2024-04-25T16:21:02.301 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T16:21:02.301 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/orig.config.yaml 2024-04-25T16:21:02.490 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T16:21:02.590 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673263' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHURfvh9sUpbhyGvSS0JiGGR/IYYvEWYs4eJ/el3XPz2AokYBYTycIZUenLNyCXK7GfuvBonxia57zMHPEBOR+8= smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBtNxWV7Iqte/HfqVxNhwBQ+C7WeEL3bN2y4Q6AVWwnLT2puphE0CJQUot2PZRoZw8RBfm5nJVxSSDr9FL10wk= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGXKQDrVO9Nh0jJ0v/bl77wWY9eodLl4k0MyzB4T9cMkGGP/YWoKRo6TTnySXkQv4FvTWd2me7hjQZ30/RgxCzA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T16:21:02.590 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T16:21:02.591 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T16:21:02.591 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T16:21:02.593 INFO:teuthology.task.internal:Checking packages... 2024-04-25T16:21:02.616 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T16:21:02.616 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T16:21:02.616 INFO:teuthology.packaging:ref: None 2024-04-25T16:21:02.617 INFO:teuthology.packaging:tag: None 2024-04-25T16:21:02.617 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:21:02.617 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:21:02.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:21:02.761 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T16:21:02.763 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T16:21:02.770 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T16:21:02.770 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T16:21:02.776 INFO:teuthology.task.internal:Saving configuration 2024-04-25T16:21:02.798 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T16:21:02.803 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T16:21:02.829 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:15:57.649947', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHURfvh9sUpbhyGvSS0JiGGR/IYYvEWYs4eJ/el3XPz2AokYBYTycIZUenLNyCXK7GfuvBonxia57zMHPEBOR+8='} 2024-04-25T16:21:02.852 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:15:57.647323', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBtNxWV7Iqte/HfqVxNhwBQ+C7WeEL3bN2y4Q6AVWwnLT2puphE0CJQUot2PZRoZw8RBfm5nJVxSSDr9FL10wk='} 2024-04-25T16:21:02.873 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:15:57.648732', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGXKQDrVO9Nh0jJ0v/bl77wWY9eodLl4k0MyzB4T9cMkGGP/YWoKRo6TTnySXkQv4FvTWd2me7hjQZ30/RgxCzA='} 2024-04-25T16:21:02.874 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T16:21:02.879 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T16:21:02.879 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T16:21:02.879 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T16:21:02.879 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T16:21:02.988 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd5bdd9d0d0>, signals=[15]) 2024-04-25T16:21:02.988 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T16:21:02.994 INFO:teuthology.task.internal:Opening connections... 2024-04-25T16:21:02.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:21:02.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:21:03.077 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-04-25T16:21:03.078 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:21:03.161 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T16:21:03.162 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:21:03.238 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T16:21:03.245 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T16:21:03.262 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T16:21:03.262 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T16:21:03.319 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:21:03.320 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:21:03.321 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T16:21:03.345 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-04-25T16:21:03.363 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-04-25T16:21:03.363 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-04-25T16:21:03.420 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="CentOS Stream" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="9" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:ID="centos" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="9" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:ANSI_COLOR="0;31" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:21:03.421 INFO:teuthology.orchestra.run.smithi116.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:21:03.422 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-04-25T16:21:03.444 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-04-25T16:21:03.462 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-04-25T16:21:03.463 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-04-25T16:21:03.519 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-04-25T16:21:03.520 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T16:21:03.521 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-04-25T16:21:03.521 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T16:21:03.521 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T16:21:03.521 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T16:21:03.521 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-04-25T16:21:03.544 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T16:21:03.555 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T16:21:03.560 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T16:21:03.561 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:21:03.563 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:21:03.565 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-04-25T16:21:03.582 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T16:21:03.589 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T16:21:03.589 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:21:03.620 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:21:03.622 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-04-25T16:21:03.726 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T16:21:03.808 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T16:21:03.833 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T16:21:03.833 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T16:21:03.834 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T16:21:03.834 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:21:03.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:21:03.851 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T16:21:03.909 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-437.el9.x86_64 2024-04-25T16:21:03.909 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-437.el9.x86_64 2024-04-25T16:21:03.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2024-04-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-04-25T16:21:08.695 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T16:21:12.712 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T16:21:14.268 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-04-25T16:21:16.189 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T16:21:21.253 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T16:21:23.134 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T16:21:23.134 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T16:21:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T16:21:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:Nothing to do. 2024-04-25T16:21:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T16:21:23.314 DEBUG:teuthology.orchestra.run.smithi112:> echo no | sudo yum reinstall kernel || true 2024-04-25T16:21:23.816 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 04:21:21 PM UTC. 2024-04-25T16:21:23.937 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-25T16:21:23.964 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-25T16:21:23.965 INFO:teuthology.orchestra.run.smithi112.stderr:Operation aborted. 2024-04-25T16:21:24.019 INFO:teuthology.orchestra.run.smithi112.stdout:Is this ok [y/N]: 2024-04-25T16:21:24.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum reinstall -y kernel || true 2024-04-25T16:21:24.515 INFO:teuthology.orchestra.run.smithi112.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 04:21:21 PM UTC. 2024-04-25T16:21:24.631 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalling: 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 255 k 2024-04-25T16:21:24.632 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 0 2024-04-25T16:21:24.633 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T16:21:24.905 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.9 MB/s | 255 kB 00:00 2024-04-25T16:21:24.906 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:21:24.906 INFO:teuthology.orchestra.run.smithi112.stdout:Total 931 kB/s | 255 kB 00:00 2024-04-25T16:21:24.928 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T16:21:25.103 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T16:21:25.103 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T16:21:25.108 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T16:21:25.109 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T16:21:25.237 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T16:21:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:21:25.331 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:21:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:21:25.952 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T16:21:26.639 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T16:21:26.639 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:21:26.639 INFO:teuthology.orchestra.run.smithi112.stdout:Reinstalled: 2024-04-25T16:21:26.639 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T16:21:26.639 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:21:26.639 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T16:21:26.830 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T16:21:26.876 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T16:21:26.877 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T16:21:26.877 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T16:21:26.877 DEBUG:teuthology.parallel:result is None 2024-04-25T16:21:26.877 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T16:21:26.884 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T16:21:26.884 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:21:26.886 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:21:26.888 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T16:21:26.906 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T16:21:26.913 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T16:21:26.924 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T16:21:26.924 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:21:26.941 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:21:26.947 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T16:21:27.047 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T16:21:27.054 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T16:21:27.054 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:21:27.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:21:27.071 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T16:21:27.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:21:27.088 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:21:27.090 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:21:27.114 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T16:21:27.134 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:21:27.143 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:21:27.146 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:21:27.165 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:21:27.175 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:21:27.177 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T16:21:27.178 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T16:21:27.186 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T16:21:27.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:21:27.188 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:21:27.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T16:21:27.246 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T16:21:27.254 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T16:21:27.255 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:21:27.257 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:21:27.287 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T16:21:27.302 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:21:27.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:21:27.406 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:21:27.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:21:27.526 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:21:27.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:21:27.589 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:21:27.633 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:21:27.679 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:21:27.761 DEBUG:teuthology.orchestra.run.smithi116:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:21:27.805 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:21:27.805 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:21:27.870 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:21:27.916 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T16:21:27.964 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:21:28.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T16:21:28.089 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:21:28.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T16:21:28.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T16:21:28.157 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-04-25T16:21:28.159 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-04-25T16:21:28.190 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:21:28.195 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:21:28.234 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:21:28.740 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T16:21:28.748 INFO:teuthology.task.internal:Starting timer... 2024-04-25T16:21:28.748 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T16:21:28.759 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T16:21:28.767 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T16:21:28.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2024-04-25T16:21:28.836 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2024-04-25T16:21:28.838 DEBUG:teuthology.orchestra.run.smithi116:> sudo service auditd rotate 2024-04-25T16:21:28.908 INFO:teuthology.orchestra.run.smithi116.stdout:Rotating logs: 2024-04-25T16:21:28.910 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-04-25T16:21:28.991 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-04-25T16:21:28.993 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T16:21:28.993 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2024-04-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2024-04-25T16:21:29.013 DEBUG:teuthology.orchestra.run.smithi116:> /usr/sbin/getenforce 2024-04-25T16:21:29.036 INFO:teuthology.orchestra.run.smithi116.stdout:Permissive 2024-04-25T16:21:29.036 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-04-25T16:21:29.077 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-04-25T16:21:29.077 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi112.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi116.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2024-04-25T16:21:29.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:21:29.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:21:29.102 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:21:29.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:21:29.130 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:21:29.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:21:29.159 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T16:21:29.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:21:29.190 DEBUG:teuthology.orchestra.run.smithi116:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:21:29.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-04-25T16:21:29.265 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T16:21:29.275 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T16:21:29.304 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T16:21:29.305 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi116.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T16:25:14.493 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi116.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-04-25T16:25:14.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T16:25:14.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:25:14.573 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T16:25:14.649 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T16:25:14.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-04-25T16:25:14.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:25:14.723 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-04-25T16:25:14.806 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-04-25T16:25:14.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T16:25:14.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T16:25:14.882 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-04-25T16:25:14.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-04-25T16:25:14.966 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T16:25:14.979 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T16:25:14.979 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:25:14.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:25:14.982 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:25:14.982 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:25:14.984 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T16:25:14.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:25:15.009 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:25:15.014 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:25:15.022 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:25:15.029 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:25:15.034 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T16:25:15.045 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2024-04-25T16:25:15.047 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T16:25:15.053 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: ntpd: command not found 2024-04-25T16:25:15.056 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2024-04-25T16:25:15.064 INFO:teuthology.orchestra.run.smithi116.stdout:506 Cannot talk to daemon 2024-04-25T16:25:15.069 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:25:15.070 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-04-25T16:25:15.078 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:25:15.080 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-04-25T16:25:15.082 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:25:15.091 INFO:teuthology.orchestra.run.smithi116.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:25:15.094 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T16:25:15.106 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T16:25:15.136 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:25:15.142 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:25:15.142 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T16:25:15.142 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.143 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.143 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.143 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.152 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:25:15.156 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:25:15.156 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-04-25T16:25:15.157 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.157 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.157 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.157 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.172 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:25:15.174 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:25:15.175 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-04-25T16:25:15.175 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.175 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.175 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.175 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:25:15.176 INFO:teuthology.run_tasks:Running task install... 2024-04-25T16:25:15.184 DEBUG:teuthology.task.install:project ceph 2024-04-25T16:25:15.184 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T16:25:15.184 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T16:25:15.185 INFO:teuthology.task.install:Using flavor: default 2024-04-25T16:25:15.190 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T16:25:15.190 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T16:25:15.191 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:25:15.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:25:15.193 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:25:15.193 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:25:15.194 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T16:25:15.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:25:15.338 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:25:15.338 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:25:15.409 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:25:15.409 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:25:15.421 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/ 2024-04-25T16:25:15.421 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3291.g994e2f20 2024-04-25T16:25:15.510 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:25:15.510 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:25:15.510 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:25:15.517 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:25:15.517 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:25:15.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:25:15.528 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T16:25:15.528 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:25:15.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T16:25:15.535 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:25:15.536 DEBUG:teuthology.orchestra.run.smithi116:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:25:15.542 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:25:15.542 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:25:15.553 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T16:25:15.553 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/994e2f20102a20fa9a27966c14b625cc01884df8/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T16:25:15.601 DEBUG:teuthology.orchestra.run.smithi116:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:25:15.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:25:15.617 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T16:25:15.674 DEBUG:teuthology.orchestra.run.smithi116:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:25:15.677 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:25:15.691 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T16:25:15.736 INFO:teuthology.orchestra.run.smithi116.stdout:check_obsoletes = 1 2024-04-25T16:25:15.738 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum clean all 2024-04-25T16:25:15.740 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2024-04-25T16:25:15.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2024-04-25T16:25:15.754 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-04-25T16:25:15.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-04-25T16:25:15.962 INFO:teuthology.orchestra.run.smithi116.stdout:48 files removed 2024-04-25T16:25:15.966 INFO:teuthology.orchestra.run.smithi112.stdout:48 files removed 2024-04-25T16:25:15.982 INFO:teuthology.orchestra.run.smithi133.stdout:48 files removed 2024-04-25T16:25:15.989 DEBUG:teuthology.orchestra.run.smithi116:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:25:15.993 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:25:16.009 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T16:25:16.651 INFO:teuthology.orchestra.run.smithi116.stdout:ceph packages for x86_64 207 kB/s | 84 kB 00:00 2024-04-25T16:25:16.675 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 206 kB/s | 84 kB 00:00 2024-04-25T16:25:16.724 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 177 kB/s | 84 kB 00:00 2024-04-25T16:25:17.059 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-04-25T16:25:17.067 INFO:teuthology.orchestra.run.smithi116.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T16:25:17.141 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-25T16:25:17.410 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-04-25T16:25:17.483 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-04-25T16:25:17.501 INFO:teuthology.orchestra.run.smithi116.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-04-25T16:25:18.237 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T16:25:18.252 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T16:25:18.296 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T16:25:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-25T16:25:20.437 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-25T16:25:20.522 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T16:25:24.214 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T16:25:24.638 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T16:25:24.751 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-25T16:25:26.086 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 16 kB 00:00 2024-04-25T16:25:26.612 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-25T16:25:27.163 INFO:teuthology.orchestra.run.smithi116.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-25T16:25:27.532 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T16:25:27.597 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-25T16:25:28.272 INFO:teuthology.orchestra.run.smithi116.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-25T16:25:32.588 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-25T16:25:32.838 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-04-25T16:25:33.281 INFO:teuthology.orchestra.run.smithi116.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T16:25:34.714 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:25:34.714 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:25:34.725 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:25:34.726 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:25:34.727 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:25:34.728 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:25:34.729 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:25:34.731 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:25:34.732 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:25:34.795 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:25:34.804 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:25:34.805 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:25:34.806 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:25:34.807 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:25:34.808 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:25:34.809 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:25:34.810 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:25:34.811 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:25:34.812 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:25:34.813 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:25:34.814 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:25:34.815 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout:======================================================================================= 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout:Install 148 Packages 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 4 Packages 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:25:34.816 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 214 M 2024-04-25T16:25:34.817 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2024-04-25T16:25:34.955 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:25:34.956 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:25:34.966 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:25:34.967 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:25:34.968 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:25:34.968 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:25:34.970 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:25:34.973 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:25:34.973 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:25:35.032 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-04-25T16:25:35.054 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-04-25T16:25:35.054 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:25:35.055 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:25:35.056 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:25:35.057 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:25:35.058 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:25:35.058 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:25:35.058 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:25:35.058 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:25:35.059 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:25:35.060 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:25:35.061 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:25:35.062 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:25:35.063 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:25:35.064 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:25:35.065 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:25:35.066 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:25:35.067 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:25:35.068 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:25:35.069 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout:Install 148 Packages 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 4 Packages 2024-04-25T16:25:35.070 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:25:35.071 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 214 M 2024-04-25T16:25:35.071 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-04-25T16:25:35.398 INFO:teuthology.orchestra.run.smithi116.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:25:35.399 INFO:teuthology.orchestra.run.smithi116.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T16:25:35.409 INFO:teuthology.orchestra.run.smithi116.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T16:25:35.410 INFO:teuthology.orchestra.run.smithi116.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T16:25:35.411 INFO:teuthology.orchestra.run.smithi116.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T16:25:35.412 INFO:teuthology.orchestra.run.smithi116.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T16:25:35.413 INFO:teuthology.orchestra.run.smithi116.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T16:25:35.416 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T16:25:35.416 INFO:teuthology.orchestra.run.smithi116.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T16:25:35.476 INFO:teuthology.orchestra.run.smithi116.stdout:Dependencies resolved. 2024-04-25T16:25:35.491 INFO:teuthology.orchestra.run.smithi116.stdout:======================================================================================= 2024-04-25T16:25:35.491 INFO:teuthology.orchestra.run.smithi116.stdout: Package Arch Version Repository Size 2024-04-25T16:25:35.491 INFO:teuthology.orchestra.run.smithi116.stdout:======================================================================================= 2024-04-25T16:25:35.491 INFO:teuthology.orchestra.run.smithi116.stdout:Installing: 2024-04-25T16:25:35.491 INFO:teuthology.orchestra.run.smithi116.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T16:25:35.491 INFO:teuthology.orchestra.run.smithi116.stdout: ceph x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 6.6 k 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.4 M 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 863 k 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 147 k 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 1.5 M 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 142 k 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 3.5 M 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 7.4 M 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 49 k 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 10 M 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 43 M 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 273 k 2024-04-25T16:25:35.492 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 762 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 224 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 27 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 18 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 32 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 756 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 127 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T16:25:35.493 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 161 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 321 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 298 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 100 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 86 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.0 M 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 169 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrading: 2024-04-25T16:25:35.494 INFO:teuthology.orchestra.run.smithi116.stdout: librados2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.3 M 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 3.1 M 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout:Installing dependencies: 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 20 M 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 34 k 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 2.3 M 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 249 k 2024-04-25T16:25:35.495 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 4.9 M 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 18 M 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3291.g994e2f20.el9 ceph-noarch 16 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 25 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 158 k 2024-04-25T16:25:35.496 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 497 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2 x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 5.2 M 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T16:25:35.497 INFO:teuthology.orchestra.run.smithi116.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 45 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common x86_64 2:19.0.0-3291.g994e2f20.el9 ceph 135 k 2024-04-25T16:25:35.498 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T16:25:35.499 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T16:25:35.500 INFO:teuthology.orchestra.run.smithi116.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T16:25:35.501 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T16:25:35.502 INFO:teuthology.orchestra.run.smithi116.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout:Installing weak dependencies: 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction Summary 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout:======================================================================================= 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout:Install 148 Packages 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout:Upgrade 4 Packages 2024-04-25T16:25:35.503 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:25:35.504 INFO:teuthology.orchestra.run.smithi116.stdout:Total download size: 214 M 2024-04-25T16:25:35.504 INFO:teuthology.orchestra.run.smithi116.stdout:Downloading Packages: 2024-04-25T16:25:36.422 INFO:teuthology.orchestra.run.smithi112.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T16:25:36.556 INFO:teuthology.orchestra.run.smithi133.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-25T16:25:36.647 INFO:teuthology.orchestra.run.smithi112.stdout:(2/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 3.8 MB/s | 863 kB 00:00 2024-04-25T16:25:36.731 INFO:teuthology.orchestra.run.smithi112.stdout:(3/152): ceph-immutable-object-cache-19.0.0-329 1.7 MB/s | 147 kB 00:00 2024-04-25T16:25:36.840 INFO:teuthology.orchestra.run.smithi116.stdout:(1/152): ceph-19.0.0-3291.g994e2f20.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-25T16:25:37.273 INFO:teuthology.orchestra.run.smithi112.stdout:(4/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 4.2 MB/s | 2.3 MB 00:00 2024-04-25T16:25:37.514 INFO:teuthology.orchestra.run.smithi133.stdout:(2/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-25T16:25:37.548 INFO:teuthology.orchestra.run.smithi133.stdout:(3/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 870 kB/s | 863 kB 00:00 2024-04-25T16:25:37.581 INFO:teuthology.orchestra.run.smithi112.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 4.9 MB/s | 1.5 MB 00:00 2024-04-25T16:25:37.598 INFO:teuthology.orchestra.run.smithi133.stdout:(4/152): ceph-immutable-object-cache-19.0.0-329 1.7 MB/s | 147 kB 00:00 2024-04-25T16:25:38.057 INFO:teuthology.orchestra.run.smithi116.stdout:(2/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 3.1 MB/s | 5.4 MB 00:01 2024-04-25T16:25:38.065 INFO:teuthology.orchestra.run.smithi133.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-04-25T16:25:38.190 INFO:teuthology.orchestra.run.smithi116.stdout:(3/152): ceph-fuse-19.0.0-3291.g994e2f20.el9.x8 639 kB/s | 863 kB 00:01 2024-04-25T16:25:38.341 INFO:teuthology.orchestra.run.smithi116.stdout:(4/152): ceph-immutable-object-cache-19.0.0-329 518 kB/s | 147 kB 00:00 2024-04-25T16:25:38.449 INFO:teuthology.orchestra.run.smithi112.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 5.6 MB/s | 4.9 MB 00:00 2024-04-25T16:25:38.799 INFO:teuthology.orchestra.run.smithi116.stdout:(5/152): ceph-mgr-19.0.0-3291.g994e2f20.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-25T16:25:39.016 INFO:teuthology.orchestra.run.smithi112.stdout:(7/152): ceph-base-19.0.0-3291.g994e2f20.el9.x8 1.8 MB/s | 5.4 MB 00:03 2024-04-25T16:25:39.808 INFO:teuthology.orchestra.run.smithi133.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 2.8 MB/s | 4.9 MB 00:01 2024-04-25T16:25:39.957 INFO:teuthology.orchestra.run.smithi133.stdout:(7/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 960 kB/s | 2.3 MB 00:02 2024-04-25T16:25:39.983 INFO:teuthology.orchestra.run.smithi116.stdout:(6/152): ceph-mon-19.0.0-3291.g994e2f20.el9.x86 4.1 MB/s | 4.9 MB 00:01 2024-04-25T16:25:40.641 INFO:teuthology.orchestra.run.smithi112.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 4.2 MB/s | 20 MB 00:04 2024-04-25T16:25:40.724 INFO:teuthology.orchestra.run.smithi112.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 303 kB/s | 25 kB 00:00 2024-04-25T16:25:40.816 INFO:teuthology.orchestra.run.smithi116.stdout:(7/152): ceph-mds-19.0.0-3291.g994e2f20.el9.x86 881 kB/s | 2.3 MB 00:02 2024-04-25T16:25:41.457 INFO:teuthology.orchestra.run.smithi133.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 3.8 MB/s | 20 MB 00:05 2024-04-25T16:25:41.557 INFO:teuthology.orchestra.run.smithi133.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 252 kB/s | 25 kB 00:00 2024-04-25T16:25:42.541 INFO:teuthology.orchestra.run.smithi116.stdout:(8/152): ceph-common-19.0.0-3291.g994e2f20.el9. 3.2 MB/s | 20 MB 00:06 2024-04-25T16:25:42.633 INFO:teuthology.orchestra.run.smithi116.stdout:(9/152): ceph-selinux-19.0.0-3291.g994e2f20.el9 275 kB/s | 25 kB 00:00 2024-04-25T16:25:43.891 INFO:teuthology.orchestra.run.smithi112.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.3 MB/s | 18 MB 00:05 2024-04-25T16:25:44.017 INFO:teuthology.orchestra.run.smithi112.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.8 MB/s | 224 kB 00:00 2024-04-25T16:25:44.099 INFO:teuthology.orchestra.run.smithi112.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 324 kB/s | 27 kB 00:00 2024-04-25T16:25:44.125 INFO:teuthology.orchestra.run.smithi116.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 4.3 MB/s | 18 MB 00:04 2024-04-25T16:25:44.174 INFO:teuthology.orchestra.run.smithi112.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 424 kB/s | 32 kB 00:00 2024-04-25T16:25:44.275 INFO:teuthology.orchestra.run.smithi116.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.5 MB/s | 224 kB 00:00 2024-04-25T16:25:44.350 INFO:teuthology.orchestra.run.smithi112.stdout:(14/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 1.9 MB/s | 10 MB 00:05 2024-04-25T16:25:44.351 INFO:teuthology.orchestra.run.smithi116.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 361 kB/s | 27 kB 00:00 2024-04-25T16:25:44.442 INFO:teuthology.orchestra.run.smithi116.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 347 kB/s | 32 kB 00:00 2024-04-25T16:25:44.459 INFO:teuthology.orchestra.run.smithi112.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.4 MB/s | 158 kB 00:00 2024-04-25T16:25:44.484 INFO:teuthology.orchestra.run.smithi112.stdout:(16/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.4 MB/s | 756 kB 00:00 2024-04-25T16:25:44.568 INFO:teuthology.orchestra.run.smithi112.stdout:(17/152): librados-devel-19.0.0-3291.g994e2f20. 1.1 MB/s | 127 kB 00:00 2024-04-25T16:25:44.643 INFO:teuthology.orchestra.run.smithi116.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 3.7 MB/s | 756 kB 00:00 2024-04-25T16:25:44.701 INFO:teuthology.orchestra.run.smithi112.stdout:(18/152): libradosstriper1-19.0.0-3291.g994e2f2 2.2 MB/s | 497 kB 00:00 2024-04-25T16:25:44.709 INFO:teuthology.orchestra.run.smithi116.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 2.3 MB/s | 158 kB 00:00 2024-04-25T16:25:44.768 INFO:teuthology.orchestra.run.smithi112.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 677 kB/s | 45 kB 00:00 2024-04-25T16:25:44.777 INFO:teuthology.orchestra.run.smithi116.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.9 MB/s | 127 kB 00:00 2024-04-25T16:25:44.835 INFO:teuthology.orchestra.run.smithi112.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 2.0 MB/s | 135 kB 00:00 2024-04-25T16:25:44.849 INFO:teuthology.orchestra.run.smithi133.stdout:(10/152): ceph-osd-19.0.0-3291.g994e2f20.el9.x8 3.5 MB/s | 18 MB 00:05 2024-04-25T16:25:44.910 INFO:teuthology.orchestra.run.smithi116.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 3.6 MB/s | 497 kB 00:00 2024-04-25T16:25:44.919 INFO:teuthology.orchestra.run.smithi112.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.9 MB/s | 161 kB 00:00 2024-04-25T16:25:44.982 INFO:teuthology.orchestra.run.smithi133.stdout:(11/152): cephfs-mirror-19.0.0-3291.g994e2f20.e 1.6 MB/s | 224 kB 00:00 2024-04-25T16:25:45.082 INFO:teuthology.orchestra.run.smithi133.stdout:(12/152): cephfs-shell-19.0.0-3291.g994e2f20.el 270 kB/s | 27 kB 00:00 2024-04-25T16:25:45.086 INFO:teuthology.orchestra.run.smithi112.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.9 MB/s | 321 kB 00:00 2024-04-25T16:25:45.157 INFO:teuthology.orchestra.run.smithi133.stdout:(13/152): libcephfs-devel-19.0.0-3291.g994e2f20 423 kB/s | 32 kB 00:00 2024-04-25T16:25:45.236 INFO:teuthology.orchestra.run.smithi112.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.9 MB/s | 298 kB 00:00 2024-04-25T16:25:45.328 INFO:teuthology.orchestra.run.smithi112.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.1 MB/s | 100 kB 00:00 2024-04-25T16:25:45.429 INFO:teuthology.orchestra.run.smithi112.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 864 kB/s | 86 kB 00:00 2024-04-25T16:25:45.475 INFO:teuthology.orchestra.run.smithi133.stdout:(14/152): libcephfs2-19.0.0-3291.g994e2f20.el9. 2.3 MB/s | 756 kB 00:00 2024-04-25T16:25:45.575 INFO:teuthology.orchestra.run.smithi133.stdout:(15/152): libcephsqlite-19.0.0-3291.g994e2f20.e 1.5 MB/s | 158 kB 00:00 2024-04-25T16:25:45.675 INFO:teuthology.orchestra.run.smithi133.stdout:(16/152): librados-devel-19.0.0-3291.g994e2f20. 1.2 MB/s | 127 kB 00:00 2024-04-25T16:25:45.867 INFO:teuthology.orchestra.run.smithi133.stdout:(17/152): libradosstriper1-19.0.0-3291.g994e2f2 2.5 MB/s | 497 kB 00:00 2024-04-25T16:25:46.285 INFO:teuthology.orchestra.run.smithi116.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 3.8 MB/s | 5.2 MB 00:01 2024-04-25T16:25:46.385 INFO:teuthology.orchestra.run.smithi116.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 452 kB/s | 45 kB 00:00 2024-04-25T16:25:46.454 INFO:teuthology.orchestra.run.smithi112.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.0 MB/s | 3.0 MB 00:01 2024-04-25T16:25:46.519 INFO:teuthology.orchestra.run.smithi116.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.0 MB/s | 135 kB 00:00 2024-04-25T16:25:46.546 INFO:teuthology.orchestra.run.smithi112.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 1.8 MB/s | 169 kB 00:00 2024-04-25T16:25:46.611 INFO:teuthology.orchestra.run.smithi116.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T16:25:46.638 INFO:teuthology.orchestra.run.smithi112.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 372 kB/s | 34 kB 00:00 2024-04-25T16:25:46.738 INFO:teuthology.orchestra.run.smithi112.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.4 MB/s | 142 kB 00:00 2024-04-25T16:25:46.778 INFO:teuthology.orchestra.run.smithi116.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.9 MB/s | 321 kB 00:00 2024-04-25T16:25:46.953 INFO:teuthology.orchestra.run.smithi116.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.7 MB/s | 298 kB 00:00 2024-04-25T16:25:47.053 INFO:teuthology.orchestra.run.smithi116.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 996 kB/s | 100 kB 00:00 2024-04-25T16:25:47.154 INFO:teuthology.orchestra.run.smithi116.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 863 kB/s | 86 kB 00:00 2024-04-25T16:25:47.234 INFO:teuthology.orchestra.run.smithi133.stdout:(18/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 3.8 MB/s | 5.2 MB 00:01 2024-04-25T16:25:47.326 INFO:teuthology.orchestra.run.smithi133.stdout:(19/152): python3-ceph-argparse-19.0.0-3291.g99 493 kB/s | 45 kB 00:00 2024-04-25T16:25:47.426 INFO:teuthology.orchestra.run.smithi133.stdout:(20/152): python3-ceph-common-19.0.0-3291.g994e 1.3 MB/s | 135 kB 00:00 2024-04-25T16:25:47.518 INFO:teuthology.orchestra.run.smithi133.stdout:(21/152): python3-cephfs-19.0.0-3291.g994e2f20. 1.7 MB/s | 161 kB 00:00 2024-04-25T16:25:47.547 INFO:teuthology.orchestra.run.smithi112.stdout:(30/152): librgw2-19.0.0-3291.g994e2f20.el9.x86 1.7 MB/s | 5.2 MB 00:02 2024-04-25T16:25:47.685 INFO:teuthology.orchestra.run.smithi133.stdout:(22/152): python3-rados-19.0.0-3291.g994e2f20.e 1.9 MB/s | 321 kB 00:00 2024-04-25T16:25:47.844 INFO:teuthology.orchestra.run.smithi133.stdout:(23/152): python3-rbd-19.0.0-3291.g994e2f20.el9 1.8 MB/s | 298 kB 00:00 2024-04-25T16:25:47.927 INFO:teuthology.orchestra.run.smithi133.stdout:(24/152): python3-rgw-19.0.0-3291.g994e2f20.el9 1.2 MB/s | 100 kB 00:00 2024-04-25T16:25:47.930 INFO:teuthology.orchestra.run.smithi112.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 3.0 MB/s | 3.5 MB 00:01 2024-04-25T16:25:48.019 INFO:teuthology.orchestra.run.smithi133.stdout:(25/152): rbd-fuse-19.0.0-3291.g994e2f20.el9.x8 944 kB/s | 86 kB 00:00 2024-04-25T16:25:48.054 INFO:teuthology.orchestra.run.smithi116.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.4 MB/s | 3.0 MB 00:00 2024-04-25T16:25:48.097 INFO:teuthology.orchestra.run.smithi112.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.5 MB/s | 249 kB 00:00 2024-04-25T16:25:48.197 INFO:teuthology.orchestra.run.smithi112.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 493 kB/s | 49 kB 00:00 2024-04-25T16:25:48.298 INFO:teuthology.orchestra.run.smithi112.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 164 kB/s | 16 kB 00:00 2024-04-25T16:25:48.473 INFO:teuthology.orchestra.run.smithi112.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.5 MB/s | 273 kB 00:00 2024-04-25T16:25:48.621 INFO:teuthology.orchestra.run.smithi116.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 299 kB/s | 169 kB 00:00 2024-04-25T16:25:48.713 INFO:teuthology.orchestra.run.smithi116.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 374 kB/s | 34 kB 00:00 2024-04-25T16:25:48.865 INFO:teuthology.orchestra.run.smithi112.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-25T16:25:48.870 INFO:teuthology.orchestra.run.smithi133.stdout:(26/152): rbd-mirror-19.0.0-3291.g994e2f20.el9. 3.6 MB/s | 3.0 MB 00:00 2024-04-25T16:25:48.932 INFO:teuthology.orchestra.run.smithi112.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 274 kB/s | 18 kB 00:00 2024-04-25T16:25:48.954 INFO:teuthology.orchestra.run.smithi133.stdout:(27/152): rbd-nbd-19.0.0-3291.g994e2f20.el9.x86 2.0 MB/s | 169 kB 00:00 2024-04-25T16:25:49.029 INFO:teuthology.orchestra.run.smithi133.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3291.g 457 kB/s | 34 kB 00:00 2024-04-25T16:25:49.040 INFO:teuthology.orchestra.run.smithi112.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T16:25:49.112 INFO:teuthology.orchestra.run.smithi133.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 1.7 MB/s | 142 kB 00:00 2024-04-25T16:25:49.115 INFO:teuthology.orchestra.run.smithi112.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2024-04-25T16:25:49.158 INFO:teuthology.orchestra.run.smithi112.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T16:25:49.163 INFO:teuthology.orchestra.run.smithi116.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3291.g994e2f2 316 kB/s | 142 kB 00:00 2024-04-25T16:25:49.200 INFO:teuthology.orchestra.run.smithi112.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T16:25:49.266 INFO:teuthology.orchestra.run.smithi112.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T16:25:49.292 INFO:teuthology.orchestra.run.smithi112.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 4.1 MB/s | 106 kB 00:00 2024-04-25T16:25:49.326 INFO:teuthology.orchestra.run.smithi112.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T16:25:49.368 INFO:teuthology.orchestra.run.smithi112.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T16:25:49.410 INFO:teuthology.orchestra.run.smithi112.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-25T16:25:49.452 INFO:teuthology.orchestra.run.smithi112.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 16 MB/s | 689 kB 00:00 2024-04-25T16:25:49.510 INFO:teuthology.orchestra.run.smithi112.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T16:25:49.544 INFO:teuthology.orchestra.run.smithi112.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T16:25:49.577 INFO:teuthology.orchestra.run.smithi112.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 648 kB/s | 21 kB 00:00 2024-04-25T16:25:49.619 INFO:teuthology.orchestra.run.smithi112.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T16:25:49.653 INFO:teuthology.orchestra.run.smithi112.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 988 kB/s | 33 kB 00:00 2024-04-25T16:25:49.688 INFO:teuthology.orchestra.run.smithi133.stdout:(30/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 1.0 MB/s | 10 MB 00:09 2024-04-25T16:25:49.778 INFO:teuthology.orchestra.run.smithi112.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T16:25:49.812 INFO:teuthology.orchestra.run.smithi112.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 551 kB/s | 18 kB 00:00 2024-04-25T16:25:49.845 INFO:teuthology.orchestra.run.smithi112.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 334 kB/s | 11 kB 00:00 2024-04-25T16:25:49.938 INFO:teuthology.orchestra.run.smithi112.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-25T16:25:49.971 INFO:teuthology.orchestra.run.smithi112.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T16:25:50.030 INFO:teuthology.orchestra.run.smithi112.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T16:25:50.072 INFO:teuthology.orchestra.run.smithi112.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 14 MB/s | 582 kB 00:00 2024-04-25T16:25:50.114 INFO:teuthology.orchestra.run.smithi112.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 611 kB/s | 25 kB 00:00 2024-04-25T16:25:50.156 INFO:teuthology.orchestra.run.smithi112.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T16:25:50.204 INFO:teuthology.orchestra.run.smithi133.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 3.2 MB/s | 3.5 MB 00:01 2024-04-25T16:25:50.223 INFO:teuthology.orchestra.run.smithi112.stdout:(62/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-25T16:25:50.290 INFO:teuthology.orchestra.run.smithi112.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-25T16:25:50.349 INFO:teuthology.orchestra.run.smithi112.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-04-25T16:25:50.354 INFO:teuthology.orchestra.run.smithi133.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 1.6 MB/s | 249 kB 00:00 2024-04-25T16:25:50.438 INFO:teuthology.orchestra.run.smithi133.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 590 kB/s | 49 kB 00:00 2024-04-25T16:25:50.522 INFO:teuthology.orchestra.run.smithi133.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 198 kB/s | 16 kB 00:00 2024-04-25T16:25:50.532 INFO:teuthology.orchestra.run.smithi112.stdout:(65/152): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-04-25T16:25:50.674 INFO:teuthology.orchestra.run.smithi112.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-04-25T16:25:50.689 INFO:teuthology.orchestra.run.smithi133.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.6 MB/s | 273 kB 00:00 2024-04-25T16:25:50.749 INFO:teuthology.orchestra.run.smithi112.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-25T16:25:50.808 INFO:teuthology.orchestra.run.smithi112.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-04-25T16:25:50.850 INFO:teuthology.orchestra.run.smithi112.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 3.1 MB/s | 133 kB 00:00 2024-04-25T16:25:51.014 INFO:teuthology.orchestra.run.smithi133.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 2.3 MB/s | 762 kB 00:00 2024-04-25T16:25:51.067 INFO:teuthology.orchestra.run.smithi112.stdout:(70/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-25T16:25:51.106 INFO:teuthology.orchestra.run.smithi133.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 199 kB/s | 18 kB 00:00 2024-04-25T16:25:51.134 INFO:teuthology.orchestra.run.smithi112.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T16:25:51.168 INFO:teuthology.orchestra.run.smithi112.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 2.6 MB/s | 87 kB 00:00 2024-04-25T16:25:51.215 INFO:teuthology.orchestra.run.smithi133.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T16:25:51.368 INFO:teuthology.orchestra.run.smithi112.stdout:(73/152): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-25T16:25:51.410 INFO:teuthology.orchestra.run.smithi112.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-04-25T16:25:51.423 INFO:teuthology.orchestra.run.smithi133.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-25T16:25:51.427 INFO:teuthology.orchestra.run.smithi116.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3291.g994e2 1.6 MB/s | 3.5 MB 00:02 2024-04-25T16:25:51.452 INFO:teuthology.orchestra.run.smithi112.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 5.8 MB/s | 248 kB 00:00 2024-04-25T16:25:51.486 INFO:teuthology.orchestra.run.smithi112.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 770 kB/s | 26 kB 00:00 2024-04-25T16:25:51.498 INFO:teuthology.orchestra.run.smithi133.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-04-25T16:25:51.519 INFO:teuthology.orchestra.run.smithi112.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 577 kB/s | 19 kB 00:00 2024-04-25T16:25:51.553 INFO:teuthology.orchestra.run.smithi112.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 5.0 MB/s | 172 kB 00:00 2024-04-25T16:25:51.557 INFO:teuthology.orchestra.run.smithi133.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.3 MB/s | 253 kB 00:00 2024-04-25T16:25:51.586 INFO:teuthology.orchestra.run.smithi112.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 1.0 MB/s | 35 kB 00:00 2024-04-25T16:25:51.691 INFO:teuthology.orchestra.run.smithi133.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-04-25T16:25:51.733 INFO:teuthology.orchestra.run.smithi133.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T16:25:51.762 INFO:teuthology.orchestra.run.smithi112.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T16:25:51.774 INFO:teuthology.orchestra.run.smithi133.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T16:25:51.804 INFO:teuthology.orchestra.run.smithi112.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-04-25T16:25:51.816 INFO:teuthology.orchestra.run.smithi133.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T16:25:51.846 INFO:teuthology.orchestra.run.smithi112.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-04-25T16:25:51.867 INFO:teuthology.orchestra.run.smithi133.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T16:25:51.879 INFO:teuthology.orchestra.run.smithi112.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2024-04-25T16:25:51.913 INFO:teuthology.orchestra.run.smithi112.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-04-25T16:25:51.942 INFO:teuthology.orchestra.run.smithi133.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2024-04-25T16:25:51.955 INFO:teuthology.orchestra.run.smithi112.stdout:(85/152): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-04-25T16:25:51.997 INFO:teuthology.orchestra.run.smithi112.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-04-25T16:25:52.005 INFO:teuthology.orchestra.run.smithi116.stdout:(31/152): ceph-radosgw-19.0.0-3291.g994e2f20.el 926 kB/s | 10 MB 00:11 2024-04-25T16:25:52.034 INFO:teuthology.orchestra.run.smithi133.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-04-25T16:25:52.039 INFO:teuthology.orchestra.run.smithi112.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-04-25T16:25:52.068 INFO:teuthology.orchestra.run.smithi133.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T16:25:52.081 INFO:teuthology.orchestra.run.smithi112.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-04-25T16:25:52.101 INFO:teuthology.orchestra.run.smithi133.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 646 kB/s | 21 kB 00:00 2024-04-25T16:25:52.114 INFO:teuthology.orchestra.run.smithi112.stdout:(89/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-25T16:25:52.152 INFO:teuthology.orchestra.run.smithi133.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T16:25:52.202 INFO:teuthology.orchestra.run.smithi133.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 654 kB/s | 33 kB 00:00 2024-04-25T16:25:52.431 INFO:teuthology.orchestra.run.smithi116.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3291.g99 586 kB/s | 249 kB 00:00 2024-04-25T16:25:52.477 INFO:teuthology.orchestra.run.smithi133.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-04-25T16:25:52.511 INFO:teuthology.orchestra.run.smithi133.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 551 kB/s | 18 kB 00:00 2024-04-25T16:25:52.525 INFO:teuthology.orchestra.run.smithi116.stdout:(33/152): ceph-mgr-rook-19.0.0-3291.g994e2f20.e 527 kB/s | 49 kB 00:00 2024-04-25T16:25:52.544 INFO:teuthology.orchestra.run.smithi133.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-04-25T16:25:52.586 INFO:teuthology.orchestra.run.smithi133.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-25T16:25:52.617 INFO:teuthology.orchestra.run.smithi116.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3291.g9 179 kB/s | 16 kB 00:00 2024-04-25T16:25:52.620 INFO:teuthology.orchestra.run.smithi133.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T16:25:52.679 INFO:teuthology.orchestra.run.smithi133.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T16:25:52.848 INFO:teuthology.orchestra.run.smithi112.stdout:(90/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 3.6 MB/s | 43 MB 00:12 2024-04-25T16:25:52.875 INFO:teuthology.orchestra.run.smithi116.stdout:(35/152): ceph-volume-19.0.0-3291.g994e2f20.el9 1.0 MB/s | 273 kB 00:00 2024-04-25T16:25:53.007 INFO:teuthology.orchestra.run.smithi112.stdout:(91/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-25T16:25:53.024 INFO:teuthology.orchestra.run.smithi112.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 242 kB/s | 42 kB 00:00 2024-04-25T16:25:53.040 INFO:teuthology.orchestra.run.smithi112.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-25T16:25:53.054 INFO:teuthology.orchestra.run.smithi133.stdout:(59/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 3.8 MB/s | 43 MB 00:11 2024-04-25T16:25:53.066 INFO:teuthology.orchestra.run.smithi112.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-25T16:25:53.087 INFO:teuthology.orchestra.run.smithi133.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 1.4 MB/s | 582 kB 00:00 2024-04-25T16:25:53.091 INFO:teuthology.orchestra.run.smithi112.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-25T16:25:53.113 INFO:teuthology.orchestra.run.smithi133.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 445 kB/s | 25 kB 00:00 2024-04-25T16:25:53.116 INFO:teuthology.orchestra.run.smithi112.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-25T16:25:53.138 INFO:teuthology.orchestra.run.smithi133.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-04-25T16:25:53.144 INFO:teuthology.orchestra.run.smithi112.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.7 MB/s | 305 kB 00:00 2024-04-25T16:25:53.163 INFO:teuthology.orchestra.run.smithi133.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T16:25:53.169 INFO:teuthology.orchestra.run.smithi112.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T16:25:53.188 INFO:teuthology.orchestra.run.smithi133.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T16:25:53.194 INFO:teuthology.orchestra.run.smithi112.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T16:25:53.214 INFO:teuthology.orchestra.run.smithi133.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-25T16:25:53.220 INFO:teuthology.orchestra.run.smithi112.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T16:25:53.245 INFO:teuthology.orchestra.run.smithi112.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T16:25:53.270 INFO:teuthology.orchestra.run.smithi112.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T16:25:53.304 INFO:teuthology.orchestra.run.smithi112.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T16:25:53.414 INFO:teuthology.orchestra.run.smithi133.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-25T16:25:53.501 INFO:teuthology.orchestra.run.smithi116.stdout:(36/152): cephadm-19.0.0-3291.g994e2f20.el9.noa 1.2 MB/s | 762 kB 00:00 2024-04-25T16:25:53.504 INFO:teuthology.orchestra.run.smithi112.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-04-25T16:25:53.529 INFO:teuthology.orchestra.run.smithi112.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-25T16:25:53.555 INFO:teuthology.orchestra.run.smithi112.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-04-25T16:25:53.584 INFO:teuthology.orchestra.run.smithi116.stdout:(37/152): cephfs-top-19.0.0-3291.g994e2f20.el9. 220 kB/s | 18 kB 00:00 2024-04-25T16:25:53.605 INFO:teuthology.orchestra.run.smithi112.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 522 kB/s | 26 kB 00:00 2024-04-25T16:25:53.655 INFO:teuthology.orchestra.run.smithi112.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 973 kB/s | 49 kB 00:00 2024-04-25T16:25:53.689 INFO:teuthology.orchestra.run.smithi133.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-25T16:25:53.692 INFO:teuthology.orchestra.run.smithi116.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T16:25:53.731 INFO:teuthology.orchestra.run.smithi112.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-25T16:25:53.756 INFO:teuthology.orchestra.run.smithi112.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-04-25T16:25:53.765 INFO:teuthology.orchestra.run.smithi133.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-25T16:25:53.781 INFO:teuthology.orchestra.run.smithi112.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T16:25:53.784 INFO:teuthology.orchestra.run.smithi116.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-25T16:25:53.798 INFO:teuthology.orchestra.run.smithi133.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.4 MB/s | 911 kB 00:00 2024-04-25T16:25:53.823 INFO:teuthology.orchestra.run.smithi133.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T16:25:53.826 INFO:teuthology.orchestra.run.smithi116.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T16:25:53.831 INFO:teuthology.orchestra.run.smithi112.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 11 MB/s | 838 kB 00:00 2024-04-25T16:25:53.857 INFO:teuthology.orchestra.run.smithi112.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-25T16:25:53.868 INFO:teuthology.orchestra.run.smithi116.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-04-25T16:25:53.890 INFO:teuthology.orchestra.run.smithi112.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 744 kB/s | 43 kB 00:00 2024-04-25T16:25:53.916 INFO:teuthology.orchestra.run.smithi112.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-04-25T16:25:53.941 INFO:teuthology.orchestra.run.smithi112.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T16:25:53.960 INFO:teuthology.orchestra.run.smithi116.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T16:25:53.966 INFO:teuthology.orchestra.run.smithi112.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T16:25:54.000 INFO:teuthology.orchestra.run.smithi112.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T16:25:54.007 INFO:teuthology.orchestra.run.smithi133.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-25T16:25:54.011 INFO:teuthology.orchestra.run.smithi116.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T16:25:54.041 INFO:teuthology.orchestra.run.smithi133.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-04-25T16:25:54.061 INFO:teuthology.orchestra.run.smithi116.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T16:25:54.066 INFO:teuthology.orchestra.run.smithi133.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T16:25:54.092 INFO:teuthology.orchestra.run.smithi112.stdout:(119/152): ceph-mgr-diskprediction-local-19.0.0 1.1 MB/s | 7.4 MB 00:06 2024-04-25T16:25:54.111 INFO:teuthology.orchestra.run.smithi116.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T16:25:54.117 INFO:teuthology.orchestra.run.smithi112.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-25T16:25:54.142 INFO:teuthology.orchestra.run.smithi112.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-04-25T16:25:54.192 INFO:teuthology.orchestra.run.smithi112.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2024-04-25T16:25:54.195 INFO:teuthology.orchestra.run.smithi116.stdout:(46/152): ceph-mgr-diskprediction-local-19.0.0- 2.7 MB/s | 7.4 MB 00:02 2024-04-25T16:25:54.218 INFO:teuthology.orchestra.run.smithi112.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-25T16:25:54.220 INFO:teuthology.orchestra.run.smithi116.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-25T16:25:54.243 INFO:teuthology.orchestra.run.smithi112.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-04-25T16:25:54.250 INFO:teuthology.orchestra.run.smithi133.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-04-25T16:25:54.260 INFO:teuthology.orchestra.run.smithi112.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-04-25T16:25:54.275 INFO:teuthology.orchestra.run.smithi133.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-04-25T16:25:54.286 INFO:teuthology.orchestra.run.smithi112.stdout:(126/152): python3-google-auth-2.29.0-1.el9.noa 1.0 MB/s | 206 kB 00:00 2024-04-25T16:25:54.300 INFO:teuthology.orchestra.run.smithi133.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T16:25:54.311 INFO:teuthology.orchestra.run.smithi112.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 344 kB/s | 23 kB 00:00 2024-04-25T16:25:54.317 INFO:teuthology.orchestra.run.smithi133.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-04-25T16:25:54.334 INFO:teuthology.orchestra.run.smithi133.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-04-25T16:25:54.336 INFO:teuthology.orchestra.run.smithi112.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T16:25:54.360 INFO:teuthology.orchestra.run.smithi133.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T16:25:54.361 INFO:teuthology.orchestra.run.smithi112.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-25T16:25:54.385 INFO:teuthology.orchestra.run.smithi133.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-25T16:25:54.386 INFO:teuthology.orchestra.run.smithi112.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-04-25T16:25:54.429 INFO:teuthology.orchestra.run.smithi112.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T16:25:54.453 INFO:teuthology.orchestra.run.smithi112.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-04-25T16:25:54.480 INFO:teuthology.orchestra.run.smithi112.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 671 kB/s | 79 kB 00:00 2024-04-25T16:25:54.504 INFO:teuthology.orchestra.run.smithi112.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-04-25T16:25:54.518 INFO:teuthology.orchestra.run.smithi133.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-25T16:25:54.529 INFO:teuthology.orchestra.run.smithi112.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-25T16:25:54.544 INFO:teuthology.orchestra.run.smithi133.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T16:25:54.555 INFO:teuthology.orchestra.run.smithi112.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-25T16:25:54.569 INFO:teuthology.orchestra.run.smithi133.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T16:25:54.580 INFO:teuthology.orchestra.run.smithi112.stdout:(137/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-04-25T16:25:54.594 INFO:teuthology.orchestra.run.smithi133.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T16:25:54.595 INFO:teuthology.orchestra.run.smithi116.stdout:(48/152): ceph-test-19.0.0-3291.g994e2f20.el9.x 3.6 MB/s | 43 MB 00:11 2024-04-25T16:25:54.620 INFO:teuthology.orchestra.run.smithi133.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-25T16:25:54.622 INFO:teuthology.orchestra.run.smithi112.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-25T16:25:54.629 INFO:teuthology.orchestra.run.smithi116.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-04-25T16:25:54.647 INFO:teuthology.orchestra.run.smithi112.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-25T16:25:54.653 INFO:teuthology.orchestra.run.smithi133.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T16:25:54.663 INFO:teuthology.orchestra.run.smithi116.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.1 MB/s | 943 kB 00:00 2024-04-25T16:25:54.672 INFO:teuthology.orchestra.run.smithi112.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T16:25:54.678 INFO:teuthology.orchestra.run.smithi133.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T16:25:54.698 INFO:teuthology.orchestra.run.smithi112.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-25T16:25:54.704 INFO:teuthology.orchestra.run.smithi133.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T16:25:54.704 INFO:teuthology.orchestra.run.smithi116.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-04-25T16:25:54.723 INFO:teuthology.orchestra.run.smithi112.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-25T16:25:54.729 INFO:teuthology.orchestra.run.smithi133.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T16:25:54.729 INFO:teuthology.orchestra.run.smithi116.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 324 kB/s | 21 kB 00:00 2024-04-25T16:25:54.748 INFO:teuthology.orchestra.run.smithi112.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T16:25:54.754 INFO:teuthology.orchestra.run.smithi133.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T16:25:54.763 INFO:teuthology.orchestra.run.smithi116.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-25T16:25:54.773 INFO:teuthology.orchestra.run.smithi112.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T16:25:54.789 INFO:teuthology.orchestra.run.smithi116.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-04-25T16:25:54.799 INFO:teuthology.orchestra.run.smithi112.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-04-25T16:25:54.814 INFO:teuthology.orchestra.run.smithi116.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T16:25:54.824 INFO:teuthology.orchestra.run.smithi112.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-25T16:25:54.849 INFO:teuthology.orchestra.run.smithi112.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T16:25:54.873 INFO:teuthology.orchestra.run.smithi116.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-25T16:25:54.891 INFO:teuthology.orchestra.run.smithi112.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T16:25:54.914 INFO:teuthology.orchestra.run.smithi116.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-25T16:25:54.963 INFO:teuthology.orchestra.run.smithi133.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 202 kB/s | 42 kB 00:00 2024-04-25T16:25:54.973 INFO:teuthology.orchestra.run.smithi116.stdout:(58/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-25T16:25:54.990 INFO:teuthology.orchestra.run.smithi116.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-04-25T16:25:55.065 INFO:teuthology.orchestra.run.smithi116.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-25T16:25:55.090 INFO:teuthology.orchestra.run.smithi116.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-25T16:25:55.107 INFO:teuthology.orchestra.run.smithi116.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 217 kB/s | 25 kB 00:00 2024-04-25T16:25:55.124 INFO:teuthology.orchestra.run.smithi116.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-25T16:25:55.150 INFO:teuthology.orchestra.run.smithi116.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-04-25T16:25:55.150 INFO:teuthology.orchestra.run.smithi112.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-25T16:25:55.175 INFO:teuthology.orchestra.run.smithi116.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-25T16:25:55.200 INFO:teuthology.orchestra.run.smithi116.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-04-25T16:25:55.230 INFO:teuthology.orchestra.run.smithi133.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-04-25T16:25:55.255 INFO:teuthology.orchestra.run.smithi133.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 152 kB/s | 44 kB 00:00 2024-04-25T16:25:55.280 INFO:teuthology.orchestra.run.smithi133.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T16:25:55.306 INFO:teuthology.orchestra.run.smithi133.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-04-25T16:25:55.331 INFO:teuthology.orchestra.run.smithi133.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-25T16:25:55.356 INFO:teuthology.orchestra.run.smithi133.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T16:25:55.381 INFO:teuthology.orchestra.run.smithi133.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T16:25:55.407 INFO:teuthology.orchestra.run.smithi133.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T16:25:55.432 INFO:teuthology.orchestra.run.smithi133.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T16:25:55.457 INFO:teuthology.orchestra.run.smithi133.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-04-25T16:25:55.475 INFO:teuthology.orchestra.run.smithi116.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 5.8 MB/s | 1.6 MB 00:00 2024-04-25T16:25:55.482 INFO:teuthology.orchestra.run.smithi133.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-25T16:25:55.516 INFO:teuthology.orchestra.run.smithi133.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T16:25:55.542 INFO:teuthology.orchestra.run.smithi116.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-25T16:25:55.609 INFO:teuthology.orchestra.run.smithi116.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T16:25:55.634 INFO:teuthology.orchestra.run.smithi112.stdout:(150/152): librados2-19.0.0-3291.g994e2f20.el9. 4.1 MB/s | 3.3 MB 00:00 2024-04-25T16:25:55.643 INFO:teuthology.orchestra.run.smithi116.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-25T16:25:55.668 INFO:teuthology.orchestra.run.smithi116.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T16:25:55.718 INFO:teuthology.orchestra.run.smithi116.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T16:25:55.752 INFO:teuthology.orchestra.run.smithi116.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-25T16:25:55.834 INFO:teuthology.orchestra.run.smithi112.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 2.4 MB 00:00 2024-04-25T16:25:55.902 INFO:teuthology.orchestra.run.smithi116.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-25T16:25:55.936 INFO:teuthology.orchestra.run.smithi116.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-25T16:25:56.036 INFO:teuthology.orchestra.run.smithi116.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-25T16:25:56.061 INFO:teuthology.orchestra.run.smithi116.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-04-25T16:25:56.086 INFO:teuthology.orchestra.run.smithi116.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 174 kB/s | 26 kB 00:00 2024-04-25T16:25:56.100 INFO:teuthology.orchestra.run.smithi133.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 529 kB/s | 309 kB 00:00 2024-04-25T16:25:56.112 INFO:teuthology.orchestra.run.smithi116.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 256 kB/s | 19 kB 00:00 2024-04-25T16:25:56.133 INFO:teuthology.orchestra.run.smithi133.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 844 kB/s | 548 kB 00:00 2024-04-25T16:25:56.137 INFO:teuthology.orchestra.run.smithi116.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T16:25:56.162 INFO:teuthology.orchestra.run.smithi116.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T16:25:56.196 INFO:teuthology.orchestra.run.smithi116.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-25T16:25:56.221 INFO:teuthology.orchestra.run.smithi116.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.8 MB/s | 222 kB 00:00 2024-04-25T16:25:56.233 INFO:teuthology.orchestra.run.smithi133.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 145 kB/s | 19 kB 00:00 2024-04-25T16:25:56.297 INFO:teuthology.orchestra.run.smithi116.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-04-25T16:25:56.330 INFO:teuthology.orchestra.run.smithi116.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 587 kB/s | 77 kB 00:00 2024-04-25T16:25:56.355 INFO:teuthology.orchestra.run.smithi116.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 323 kB/s | 42 kB 00:00 2024-04-25T16:25:56.380 INFO:teuthology.orchestra.run.smithi116.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-25T16:25:56.384 INFO:teuthology.orchestra.run.smithi133.stdout:(107/152): ceph-mgr-diskprediction-local-19.0.0 1.1 MB/s | 7.4 MB 00:06 2024-04-25T16:25:56.406 INFO:teuthology.orchestra.run.smithi116.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T16:25:56.409 INFO:teuthology.orchestra.run.smithi133.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-04-25T16:25:56.431 INFO:teuthology.orchestra.run.smithi116.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-25T16:25:56.456 INFO:teuthology.orchestra.run.smithi116.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 678 kB/s | 51 kB 00:00 2024-04-25T16:25:56.543 INFO:teuthology.orchestra.run.smithi133.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-04-25T16:25:56.635 INFO:teuthology.orchestra.run.smithi133.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-04-25T16:25:56.657 INFO:teuthology.orchestra.run.smithi116.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 238 kB/s | 54 kB 00:00 2024-04-25T16:25:56.677 INFO:teuthology.orchestra.run.smithi133.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-25T16:25:56.682 INFO:teuthology.orchestra.run.smithi116.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 185 kB/s | 42 kB 00:00 2024-04-25T16:25:56.759 INFO:teuthology.orchestra.run.smithi112.stdout:(152/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 1.6 MB/s | 3.1 MB 00:01 2024-04-25T16:25:56.760 INFO:teuthology.orchestra.run.smithi133.stdout:(112/152): liboath-2.6.7-2.el9.x86_64.rpm 129 kB/s | 49 kB 00:00 2024-04-25T16:25:56.762 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:25:56.763 INFO:teuthology.orchestra.run.smithi112.stdout:Total 9.8 MB/s | 214 MB 00:21 2024-04-25T16:25:56.811 INFO:teuthology.orchestra.run.smithi133.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-25T16:25:56.953 INFO:teuthology.orchestra.run.smithi133.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 226 kB/s | 43 kB 00:00 2024-04-25T16:25:56.978 INFO:teuthology.orchestra.run.smithi133.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 193 kB/s | 32 kB 00:00 2024-04-25T16:25:57.003 INFO:teuthology.orchestra.run.smithi133.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 1.6 MB/s | 548 kB 00:00 2024-04-25T16:25:57.016 INFO:teuthology.orchestra.run.smithi116.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 124 kB/s | 44 kB 00:00 2024-04-25T16:25:57.041 INFO:teuthology.orchestra.run.smithi116.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 620 kB/s | 221 kB 00:00 2024-04-25T16:25:57.079 INFO:teuthology.orchestra.run.smithi133.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 119 kB/s | 15 kB 00:00 2024-04-25T16:25:57.104 INFO:teuthology.orchestra.run.smithi133.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T16:25:57.162 INFO:teuthology.orchestra.run.smithi133.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-25T16:25:57.229 INFO:teuthology.orchestra.run.smithi133.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-04-25T16:25:57.275 INFO:teuthology.orchestra.run.smithi116.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-04-25T16:25:57.300 INFO:teuthology.orchestra.run.smithi116.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 126 kB/s | 36 kB 00:00 2024-04-25T16:25:57.313 INFO:teuthology.orchestra.run.smithi133.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-25T16:25:57.326 INFO:teuthology.orchestra.run.smithi116.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 168 kB/s | 47 kB 00:00 2024-04-25T16:25:57.350 INFO:teuthology.orchestra.run.smithi116.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T16:25:57.376 INFO:teuthology.orchestra.run.smithi116.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T16:25:57.401 INFO:teuthology.orchestra.run.smithi116.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T16:25:57.426 INFO:teuthology.orchestra.run.smithi116.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-04-25T16:25:57.438 INFO:teuthology.orchestra.run.smithi133.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 844 kB/s | 303 kB 00:00 2024-04-25T16:25:57.451 INFO:teuthology.orchestra.run.smithi116.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T16:25:57.476 INFO:teuthology.orchestra.run.smithi116.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-25T16:25:57.480 INFO:teuthology.orchestra.run.smithi133.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-25T16:25:57.505 INFO:teuthology.orchestra.run.smithi133.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 751 kB/s | 144 kB 00:00 2024-04-25T16:25:57.519 INFO:teuthology.orchestra.run.smithi116.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-25T16:25:57.539 INFO:teuthology.orchestra.run.smithi133.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 106 kB/s | 11 kB 00:00 2024-04-25T16:25:57.564 INFO:teuthology.orchestra.run.smithi133.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-04-25T16:25:57.589 INFO:teuthology.orchestra.run.smithi133.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-25T16:25:57.623 INFO:teuthology.orchestra.run.smithi133.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-25T16:25:57.656 INFO:teuthology.orchestra.run.smithi133.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-25T16:25:57.723 INFO:teuthology.orchestra.run.smithi133.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-25T16:25:57.757 INFO:teuthology.orchestra.run.smithi133.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-04-25T16:25:57.794 INFO:teuthology.orchestra.run.smithi116.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-04-25T16:25:57.807 INFO:teuthology.orchestra.run.smithi133.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-25T16:25:57.830 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2024-04-25T16:25:57.858 INFO:teuthology.orchestra.run.smithi133.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 431 kB/s | 58 kB 00:00 2024-04-25T16:25:57.883 INFO:teuthology.orchestra.run.smithi133.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T16:25:57.908 INFO:teuthology.orchestra.run.smithi133.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T16:25:57.967 INFO:teuthology.orchestra.run.smithi133.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-25T16:25:57.971 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2024-04-25T16:25:57.971 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2024-04-25T16:25:57.985 INFO:teuthology.orchestra.run.smithi116.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 608 kB/s | 309 kB 00:00 2024-04-25T16:25:58.008 INFO:teuthology.orchestra.run.smithi133.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 225 kB/s | 28 kB 00:00 2024-04-25T16:25:58.036 INFO:teuthology.orchestra.run.smithi133.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 241 kB/s | 31 kB 00:00 2024-04-25T16:25:58.103 INFO:teuthology.orchestra.run.smithi133.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-25T16:25:58.177 INFO:teuthology.orchestra.run.smithi116.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 756 kB/s | 548 kB 00:00 2024-04-25T16:25:58.203 INFO:teuthology.orchestra.run.smithi133.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 304 kB/s | 59 kB 00:00 2024-04-25T16:25:58.244 INFO:teuthology.orchestra.run.smithi116.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-25T16:25:58.303 INFO:teuthology.orchestra.run.smithi133.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-04-25T16:25:58.337 INFO:teuthology.orchestra.run.smithi133.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-25T16:25:58.362 INFO:teuthology.orchestra.run.smithi133.stdout:(143/152): python3-typing-extensions-4.4.0-2.el 197 kB/s | 51 kB 00:00 2024-04-25T16:25:58.454 INFO:teuthology.orchestra.run.smithi133.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 595 kB/s | 90 kB 00:00 2024-04-25T16:25:58.487 INFO:teuthology.orchestra.run.smithi133.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-04-25T16:25:58.513 INFO:teuthology.orchestra.run.smithi133.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 132 kB/s | 20 kB 00:00 2024-04-25T16:25:58.528 INFO:teuthology.orchestra.run.smithi116.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-04-25T16:25:58.563 INFO:teuthology.orchestra.run.smithi133.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T16:25:58.586 INFO:teuthology.orchestra.run.smithi116.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-04-25T16:25:58.664 INFO:teuthology.orchestra.run.smithi133.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-25T16:25:58.747 INFO:teuthology.orchestra.run.smithi133.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T16:25:58.864 INFO:teuthology.orchestra.run.smithi133.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T16:25:58.920 INFO:teuthology.orchestra.run.smithi116.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2024-04-25T16:25:58.954 INFO:teuthology.orchestra.run.smithi116.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-25T16:25:59.254 INFO:teuthology.orchestra.run.smithi116.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-25T16:25:59.296 INFO:teuthology.orchestra.run.smithi116.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-04-25T16:25:59.513 INFO:teuthology.orchestra.run.smithi116.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 925 kB/s | 548 kB 00:00 2024-04-25T16:25:59.530 INFO:teuthology.orchestra.run.smithi116.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 117 kB/s | 32 kB 00:00 2024-04-25T16:25:59.555 INFO:teuthology.orchestra.run.smithi116.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 58 kB/s | 15 kB 00:00 2024-04-25T16:25:59.556 INFO:teuthology.orchestra.run.smithi133.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 3.1 MB/s | 3.1 MB 00:00 2024-04-25T16:25:59.939 INFO:teuthology.orchestra.run.smithi116.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-25T16:25:59.952 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2024-04-25T16:25:59.953 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2024-04-25T16:26:00.047 INFO:teuthology.orchestra.run.smithi116.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 693 kB/s | 358 kB 00:00 2024-04-25T16:26:00.081 INFO:teuthology.orchestra.run.smithi116.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 577 kB/s | 303 kB 00:00 2024-04-25T16:26:00.206 INFO:teuthology.orchestra.run.smithi116.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-25T16:26:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 1.7 MB/s | 3.3 MB 00:01 2024-04-25T16:26:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:26:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:Total 8.4 MB/s | 214 MB 00:25 2024-04-25T16:26:00.556 INFO:teuthology.orchestra.run.smithi116.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 404 kB/s | 206 kB 00:00 2024-04-25T16:26:00.632 INFO:teuthology.orchestra.run.smithi116.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-04-25T16:26:00.740 INFO:teuthology.orchestra.run.smithi116.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-25T16:26:00.874 INFO:teuthology.orchestra.run.smithi116.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-25T16:26:00.899 INFO:teuthology.orchestra.run.smithi116.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 66 kB/s | 18 kB 00:00 2024-04-25T16:26:01.000 INFO:teuthology.orchestra.run.smithi116.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 89 kB/s | 23 kB 00:00 2024-04-25T16:26:01.141 INFO:teuthology.orchestra.run.smithi116.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 73 kB/s | 19 kB 00:00 2024-04-25T16:26:01.167 INFO:teuthology.orchestra.run.smithi116.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 73 kB/s | 20 kB 00:00 2024-04-25T16:26:01.484 INFO:teuthology.orchestra.run.smithi116.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 135 kB/s | 46 kB 00:00 2024-04-25T16:26:01.525 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-04-25T16:26:01.619 INFO:teuthology.orchestra.run.smithi116.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 174 kB/s | 79 kB 00:00 2024-04-25T16:26:01.667 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-04-25T16:26:01.667 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-04-25T16:26:01.751 INFO:teuthology.orchestra.run.smithi116.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 1.4 MB/s | 1.0 MB 00:00 2024-04-25T16:26:01.826 INFO:teuthology.orchestra.run.smithi116.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-25T16:26:02.018 INFO:teuthology.orchestra.run.smithi116.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 62 kB/s | 16 kB 00:00 2024-04-25T16:26:02.268 INFO:teuthology.orchestra.run.smithi116.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 420 kB/s | 272 kB 00:00 2024-04-25T16:26:02.294 INFO:teuthology.orchestra.run.smithi116.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 192 kB/s | 90 kB 00:00 2024-04-25T16:26:02.460 INFO:teuthology.orchestra.run.smithi116.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 64 kB/s | 28 kB 00:00 2024-04-25T16:26:02.535 INFO:teuthology.orchestra.run.smithi116.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-25T16:26:02.811 INFO:teuthology.orchestra.run.smithi116.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 365 kB/s | 188 kB 00:00 2024-04-25T16:26:02.836 INFO:teuthology.orchestra.run.smithi116.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 157 kB/s | 59 kB 00:00 2024-04-25T16:26:02.861 INFO:teuthology.orchestra.run.smithi116.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 110 kB/s | 36 kB 00:00 2024-04-25T16:26:03.153 INFO:teuthology.orchestra.run.smithi116.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 149 kB/s | 51 kB 00:00 2024-04-25T16:26:03.295 INFO:teuthology.orchestra.run.smithi116.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 207 kB/s | 90 kB 00:00 2024-04-25T16:26:03.382 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-04-25T16:26:03.383 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-04-25T16:26:03.387 INFO:teuthology.orchestra.run.smithi116.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 437 kB/s | 230 kB 00:00 2024-04-25T16:26:03.579 INFO:teuthology.orchestra.run.smithi116.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 70 kB/s | 20 kB 00:00 2024-04-25T16:26:03.730 INFO:teuthology.orchestra.run.smithi116.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 742 kB/s | 427 kB 00:00 2024-04-25T16:26:03.896 INFO:teuthology.orchestra.run.smithi116.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 377 kB/s | 192 kB 00:00 2024-04-25T16:26:04.338 INFO:teuthology.orchestra.run.smithi116.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-04-25T16:26:04.422 INFO:teuthology.orchestra.run.smithi116.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T16:26:04.547 INFO:teuthology.orchestra.run.smithi116.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 19 MB/s | 2.4 MB 00:00 2024-04-25T16:26:05.464 INFO:teuthology.orchestra.run.smithi116.stdout:(151/152): librbd1-19.0.0-3291.g994e2f20.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-04-25T16:26:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2024-04-25T16:26:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:26:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:26:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:26:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:26:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:26:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:26:06.820 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:26:06.838 INFO:teuthology.orchestra.run.smithi116.stdout:(152/152): librados2-19.0.0-3291.g994e2f20.el9. 1.1 MB/s | 3.3 MB 00:03 2024-04-25T16:26:06.843 INFO:teuthology.orchestra.run.smithi116.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:26:06.844 INFO:teuthology.orchestra.run.smithi116.stdout:Total 6.8 MB/s | 214 MB 00:31 2024-04-25T16:26:06.932 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:07.057 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:07.074 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:07.166 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:07.195 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:26:07.352 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:26:07.401 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:26:07.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:07.534 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:07.563 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:26:07.626 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:26:07.647 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:26:07.672 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:26:07.847 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:26:07.880 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:26:07.908 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction check 2024-04-25T16:26:07.987 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:26:08.049 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction check succeeded. 2024-04-25T16:26:08.049 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction test 2024-04-25T16:26:08.082 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:26:08.122 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:26:08.156 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:26:08.220 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:26:08.255 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:26:08.299 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:26:08.331 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:26:08.359 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:26:08.428 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:26:08.481 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:26:08.512 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:26:08.547 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:26:08.623 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:26:08.658 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:26:08.702 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:26:08.739 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:26:08.807 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:26:08.853 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:26:08.927 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:26:09.060 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:26:09.122 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:26:09.445 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-04-25T16:26:09.534 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:26:09.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:26:09.625 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:26:09.643 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:26:09.659 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:26:09.685 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:26:09.727 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:26:09.728 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:26:09.764 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:26:09.799 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:26:09.837 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:26:09.879 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:26:09.928 INFO:teuthology.orchestra.run.smithi116.stdout:Transaction test succeeded. 2024-04-25T16:26:09.929 INFO:teuthology.orchestra.run.smithi116.stdout:Running transaction 2024-04-25T16:26:10.053 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:26:10.070 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:26:10.123 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:26:10.169 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:10.177 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:26:10.242 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:26:10.273 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:26:10.294 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:10.310 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:26:10.329 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:10.358 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:26:10.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:26:10.420 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:10.435 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:26:10.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:26:10.478 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:26:10.602 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:26:10.614 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:26:10.663 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:26:10.667 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:26:10.691 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:10.706 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:26:10.747 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:26:10.796 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:10.825 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:26:10.841 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:26:10.881 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:26:10.896 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:26:10.918 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:26:10.921 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:26:10.942 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:26:10.946 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:26:10.971 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:26:11.045 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:26:11.134 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:26:11.168 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:26:11.172 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:26:11.203 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:26:11.231 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:26:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:26:11.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:26:11.322 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:26:11.353 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:26:11.370 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:26:11.383 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:26:11.408 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:26:11.409 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:26:11.442 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:26:11.443 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:26:11.484 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:26:11.509 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:26:11.519 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:26:11.550 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:26:11.603 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:26:11.605 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:26:11.635 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:26:11.648 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:26:11.663 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:26:11.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:26:11.727 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:26:11.733 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:26:11.794 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:26:11.824 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:26:11.825 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:26:11.868 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:26:11.882 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:26:11.943 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:26:11.979 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:26:12.013 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:26:12.042 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:26:12.111 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:26:12.156 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:26:12.230 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:26:12.365 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:26:12.417 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:26:12.593 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:26:12.637 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:26:12.671 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:26:12.770 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:26:12.828 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:26:12.916 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:26:12.947 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:26:12.948 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:26:13.029 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:26:13.058 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:26:13.100 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:26:13.124 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:26:13.139 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:26:13.154 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:26:13.180 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:26:13.231 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:26:13.348 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:26:13.408 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:26:13.466 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:26:13.536 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:26:13.567 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:26:13.604 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:26:13.659 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:26:13.703 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:26:13.746 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:26:13.788 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:26:13.877 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:26:13.923 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:26:13.969 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:26:13.993 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:26:13.999 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:26:14.040 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:26:14.135 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:26:14.174 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:26:14.213 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:26:14.239 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:26:14.264 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:26:14.339 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:26:14.465 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:26:14.521 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:26:14.557 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:26:14.592 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:26:14.631 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:26:14.663 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:26:14.692 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:26:14.718 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:26:14.743 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:26:14.785 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:26:14.819 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:26:14.906 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:26:14.949 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:26:14.980 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:26:15.028 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:26:15.125 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:26:15.182 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:26:15.225 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:26:15.300 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:26:15.328 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:26:15.361 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:26:15.683 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:26:15.724 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:26:15.798 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:26:15.829 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:26:15.864 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:26:15.906 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:26:15.949 INFO:teuthology.orchestra.run.smithi116.stdout: Preparing : 1/1 2024-04-25T16:26:15.954 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:26:15.987 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:26:16.042 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T16:26:16.088 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:26:16.089 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T16:26:16.143 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T16:26:16.145 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:26:16.177 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T16:26:16.211 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T16:26:16.233 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:26:16.253 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T16:26:16.264 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:26:16.355 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:26:16.393 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:26:16.440 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:26:16.465 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:26:16.470 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:26:16.487 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:26:16.548 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:26:16.646 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T16:26:16.712 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:16.821 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:16.846 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:16.921 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:16.951 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:26:17.102 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T16:26:17.156 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T16:26:17.192 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:17.296 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:17.327 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:26:17.336 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:26:17.389 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 14/156 2024-04-25T16:26:17.419 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:26:17.442 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T16:26:17.443 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:26:17.619 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:26:17.652 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T16:26:17.768 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T16:26:17.872 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T16:26:17.910 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T16:26:17.944 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T16:26:18.009 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T16:26:18.043 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T16:26:18.087 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T16:26:18.120 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T16:26:18.147 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T16:26:18.217 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T16:26:18.270 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T16:26:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T16:26:18.352 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T16:26:18.437 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T16:26:18.472 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 31/156 2024-04-25T16:26:18.514 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 32/156 2024-04-25T16:26:18.543 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 33/156 2024-04-25T16:26:18.614 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T16:26:18.666 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T16:26:18.668 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:26:18.736 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:18.741 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T16:26:18.749 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:26:18.775 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:18.777 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:26:18.810 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:26:18.877 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T16:26:18.936 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T16:26:19.137 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:26:19.173 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:26:19.263 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:26:19.294 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:26:19.329 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:19.330 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:26:19.437 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:26:19.468 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T16:26:19.528 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:26:19.528 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:26:19.528 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:26:19.529 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:19.565 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T16:26:19.576 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:26:19.594 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T16:26:19.645 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T16:26:19.683 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 43/156 2024-04-25T16:26:19.724 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T16:26:19.817 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:26:19.850 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:26:19.898 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T16:26:19.921 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:26:19.944 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:26:19.952 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T16:26:20.007 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T16:26:20.081 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T16:26:20.111 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T16:26:20.148 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T16:26:20.203 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T16:26:20.256 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T16:26:20.299 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T16:26:20.341 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T16:26:20.477 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T16:26:20.529 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T16:26:20.559 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T16:26:20.601 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T16:26:20.696 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T16:26:20.735 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T16:26:20.774 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T16:26:20.809 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T16:26:20.842 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T16:26:20.917 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T16:26:21.043 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T16:26:21.082 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T16:26:21.110 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T16:26:21.144 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T16:26:21.184 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T16:26:21.215 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T16:26:21.245 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T16:26:21.280 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T16:26:21.313 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T16:26:21.355 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T16:26:21.389 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T16:26:21.476 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T16:26:21.519 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T16:26:21.551 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T16:26:21.598 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T16:26:21.705 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T16:26:21.760 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T16:26:22.215 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:22.248 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:22.490 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T16:26:22.541 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T16:26:22.574 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T16:26:22.677 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T16:26:22.741 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T16:26:22.811 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:22.829 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T16:26:22.851 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:26:22.868 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T16:26:22.951 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:26:22.951 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:26:22.951 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:26:22.951 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:22.978 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:26:23.045 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T16:26:23.074 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T16:26:23.152 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T16:26:23.801 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T16:26:23.915 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T16:26:25.156 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T16:26:25.261 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T16:26:25.289 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T16:26:25.322 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T16:26:25.649 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T16:26:25.694 INFO:teuthology.orchestra.run.smithi116.stdout: Upgrading : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:26:25.767 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 99/156 2024-04-25T16:26:25.815 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 100/156 2024-04-25T16:26:25.851 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T16:26:26.344 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T16:26:26.380 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:26:26.451 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 103/156 2024-04-25T16:26:26.473 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 104/156 2024-04-25T16:26:28.740 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:28.778 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:29.335 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 105/156 2024-04-25T16:26:29.381 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:26:29.452 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 106/156 2024-04-25T16:26:29.452 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T16:26:29.452 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T16:26:29.452 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:29.482 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:26:31.019 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2024-04-25T16:26:31.020 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:31.193 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:26:31.208 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:26:31.209 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:31.209 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:26:31.209 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:32.788 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:26:32.814 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:26:32.814 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:32.814 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:26:32.814 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:26:32.814 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:26:32.814 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:33.039 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:26:33.061 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:26:33.061 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:33.061 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:26:33.061 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:26:33.061 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:26:33.061 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:33.523 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:26:33.545 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:26:33.546 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:33.546 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:26:33.546 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:26:33.546 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:26:33.546 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:33.644 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:26:33.693 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:26:33.748 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:26:33.932 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:26:33.981 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:26:34.051 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:26:34.139 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:26:34.285 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:26:34.323 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:26:34.356 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:26:34.383 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:26:34.459 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:26:34.459 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-04-25T16:26:34.459 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-04-25T16:26:34.459 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-04-25T16:26:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-04-25T16:26:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-04-25T16:26:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-04-25T16:26:34.460 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-04-25T16:26:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:34.621 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:26:34.636 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:26:34.636 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:34.636 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:26:34.636 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:34.844 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:26:34.874 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:26:34.898 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:26:34.951 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:26:34.980 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:26:36.017 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:26:36.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:26:36.225 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:26:36.250 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:26:36.250 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:36.250 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:26:36.250 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:26:36.250 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:26:36.250 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:36.476 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:26:36.498 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:26:36.498 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:36.498 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:26:36.498 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:26:36.498 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:26:36.499 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:36.968 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:26:36.991 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:26:36.991 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:36.991 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:26:36.991 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:26:36.991 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:26:36.991 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:37.022 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:26:37.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:26:37.106 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:26:37.158 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:26:37.163 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:26:37.219 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:26:37.324 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:26:37.351 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:26:37.373 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:26:37.374 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:37.374 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:26:37.374 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:26:37.374 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:26:37.374 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:37.412 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:26:37.419 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:26:37.429 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:26:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:26:37.528 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:26:37.538 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:26:37.583 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:26:37.643 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:26:37.783 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:26:37.827 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:26:37.858 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:26:37.886 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:26:38.304 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:26:38.334 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:26:38.366 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:26:38.437 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:26:38.474 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:26:38.520 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:26:38.554 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:26:38.576 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:26:38.577 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:38.577 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:26:38.577 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:26:38.577 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:26:38.577 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:38.622 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:26:38.643 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:26:38.643 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:38.643 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:26:38.643 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:38.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:26:38.711 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:26:38.711 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:38.711 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:26:38.711 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:26:38.711 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:26:38.711 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:39.023 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:26:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:26:39.045 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:39.045 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:26:39.045 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:26:39.045 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:26:39.045 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:39.520 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:26:39.551 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:26:40.518 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:26:40.551 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:26:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:26:40.820 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:26:40.845 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:26:40.867 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:26:40.867 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:40.868 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:26:40.868 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:26:40.868 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:26:40.868 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:40.912 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:26:40.923 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 107/156 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /sys 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /proc 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /mnt 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /var/tmp 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /home 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /root 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout:skipping the directory /tmp 2024-04-25T16:26:40.944 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:41.021 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:26:41.069 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:26:41.110 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:26:41.125 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 108/156 2024-04-25T16:26:41.125 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:41.125 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T16:26:41.125 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:42.004 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:26:42.039 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:26:42.062 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:26:42.062 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:42.062 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:26:42.062 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:26:42.062 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:26:42.062 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:42.107 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:26:42.128 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:26:42.128 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:42.128 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:26:42.128 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:42.191 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:26:42.212 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:26:42.212 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:42.212 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:26:42.213 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:26:42.213 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:26:42.213 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:42.525 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:26:42.546 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:26:42.546 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:42.547 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:26:42.547 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:26:42.547 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:26:42.547 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:42.688 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:26:42.713 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 109/156 2024-04-25T16:26:42.713 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:42.713 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T16:26:42.713 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:26:42.713 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T16:26:42.713 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:42.939 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:26:42.961 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 110/156 2024-04-25T16:26:42.961 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:42.961 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T16:26:42.961 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:26:42.961 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T16:26:42.961 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:43.310 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:26:43.351 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:26:43.400 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:26:43.423 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:26:43.432 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:26:43.444 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 111/156 2024-04-25T16:26:43.444 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:43.445 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T16:26:43.445 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:26:43.445 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T16:26:43.445 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:43.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:26:43.553 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T16:26:43.601 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T16:26:43.648 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:26:43.657 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T16:26:43.689 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:26:43.719 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:26:43.842 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T16:26:43.890 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T16:26:43.959 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T16:26:44.040 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:26:44.179 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:26:44.215 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T16:26:44.247 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T16:26:44.274 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T16:26:44.306 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:44.342 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:44.361 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:44.727 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:26:44.756 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 123/156 2024-04-25T16:26:44.788 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 124/156 2024-04-25T16:26:44.850 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 125/156 2024-04-25T16:26:44.887 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:26:45.190 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:26:45.272 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:26:45.313 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:26:45.394 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:26:45.427 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:26:45.427 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:26:45.494 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:26:45.519 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:26:45.519 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:26:45.617 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:26:45.684 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:45.931 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 126/156 2024-04-25T16:26:45.973 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:26:46.820 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:26:46.862 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:26:46.902 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:26:46.934 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:26:46.944 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 127/156 2024-04-25T16:26:46.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:26:46.973 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:26:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 128/156 2024-04-25T16:26:47.144 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:26:47.191 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:26:47.221 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:26:47.239 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 129/156 2024-04-25T16:26:47.267 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:26:47.290 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 130/156 2024-04-25T16:26:47.290 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:47.290 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T16:26:47.290 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:26:47.290 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T16:26:47.290 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:47.334 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:26:47.344 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 131/156 2024-04-25T16:26:47.444 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 132/156 2024-04-25T16:26:47.491 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 133/156 2024-04-25T16:26:47.819 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:47.860 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:47.880 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:48.429 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T16:26:48.461 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:26:48.483 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 135/156 2024-04-25T16:26:48.483 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:48.483 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T16:26:48.483 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:26:48.483 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T16:26:48.483 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:48.529 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:26:48.550 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3291.g994e2 136/156 2024-04-25T16:26:48.550 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:48.551 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T16:26:48.551 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:48.596 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:26:48.617 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 137/156 2024-04-25T16:26:48.617 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:48.617 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T16:26:48.618 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:26:48.618 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T16:26:48.618 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:48.718 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:26:48.806 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:26:48.847 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:26:48.930 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:26:48.938 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:26:48.951 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 138/156 2024-04-25T16:26:48.951 INFO:teuthology.orchestra.run.smithi116.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T16:26:48.951 INFO:teuthology.orchestra.run.smithi116.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T16:26:48.951 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:26:48.952 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T16:26:48.952 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:26:48.969 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:26:48.969 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:26:49.052 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:26:49.094 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:26:49.094 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:26:49.188 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:26:49.260 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:26:51.681 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:26:51.682 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:26:51.683 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:26:51.684 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:26:51.685 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:26:51.686 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:26:51.687 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:26:51.688 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:26:51.689 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:26:51.690 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:26:51.691 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:26:51.692 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:26:51.693 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:26:51.694 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:26:51.695 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:26:51.696 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:26:51.697 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:26:53.217 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 139/156 2024-04-25T16:26:53.258 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 140/156 2024-04-25T16:26:53.299 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 141/156 2024-04-25T16:26:53.331 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T16:26:53.358 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T16:26:53.545 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 144/156 2024-04-25T16:26:53.588 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T16:26:53.617 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 146/156 2024-04-25T16:26:53.747 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:26:53.748 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.749 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:26:53.750 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:26:53.751 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:26:53.752 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:26:53.753 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:26:53.754 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:26:53.755 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:26:53.756 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:26:53.757 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.758 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:26:53.759 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:26:53.760 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2024-04-25T16:26:54.136 DEBUG:teuthology.parallel:result is None 2024-04-25T16:26:54.217 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:54.268 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T16:26:54.934 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:26:54.935 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:26:54.936 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:26:54.947 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:26:54.948 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:26:54.949 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:26:54.950 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:26:54.951 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:26:54.952 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:26:54.953 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:26:54.954 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:26:54.955 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:26:54.956 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:26:54.957 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:26:54.958 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:26:55.107 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 148/156 2024-04-25T16:26:55.186 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T16:26:55.227 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T16:26:55.309 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T16:26:55.340 INFO:teuthology.orchestra.run.smithi116.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T16:26:55.340 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:26:55.424 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T16:26:55.457 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:26:55.457 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:26:55.544 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T16:26:55.614 INFO:teuthology.orchestra.run.smithi116.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:57.025 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:26:57.026 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.027 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:26:57.028 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:26:57.029 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:26:57.030 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:26:57.031 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:26:57.032 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:26:57.033 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:26:57.034 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:26:57.035 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.036 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:26:57.037 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:26:57.038 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:26:57.038 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:26:57.038 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-04-25T16:26:57.383 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:01.548 INFO:teuthology.orchestra.run.smithi116.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 1/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 3/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 4/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3291.g994e2 5/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 6/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 7/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 8/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 9/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 10/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 11/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 12/156 2024-04-25T16:27:01.549 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 13/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 14/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_ 15/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 16/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 17/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_6 18/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86 19/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 20/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el 21/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ceph-common-2:19.0.0-3291.g994e2f20.el9. 22/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_6 23/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 24/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 25/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 26/156 2024-04-25T16:27:01.550 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 27/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 28/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 29/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20. 30/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noa 31/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.n 32/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3291.g994 33/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el 34/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 35/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.e 36/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 37/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 38/156 2024-04-25T16:27:01.563 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 39/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T16:27:01.564 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T16:27:01.565 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T16:27:01.566 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T16:27:01.567 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T16:27:01.568 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T16:27:01.569 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T16:27:01.570 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 149/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 151/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T16:27:01.571 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout:Upgraded: 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: librados2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: librbd1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout:Installed: 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T16:27:03.583 INFO:teuthology.orchestra.run.smithi116.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-grafana-dashboards-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-immutable-object-cache-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mds-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.584 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-rook-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mon-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-osd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-prometheus-alerts-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-radosgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-selinux-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-test-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-volume-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: cephadm-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.585 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-top-2:19.0.0-3291.g994e2f20.el9.noarch 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T16:27:03.586 INFO:teuthology.orchestra.run.smithi116.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libcephfs2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libcephsqlite-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: librados-devel-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.587 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: librgw2-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T16:27:03.588 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-common-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cephfs-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.589 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T16:27:03.590 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T16:27:03.591 INFO:teuthology.orchestra.run.smithi116.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T16:27:03.592 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rgw-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T16:27:03.593 INFO:teuthology.orchestra.run.smithi116.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-fuse-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-mirror-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.594 INFO:teuthology.orchestra.run.smithi116.stdout: rbd-nbd-2:19.0.0-3291.g994e2f20.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-04-25T16:27:03.595 INFO:teuthology.orchestra.run.smithi116.stdout:Complete! 2024-04-25T16:27:03.991 DEBUG:teuthology.parallel:result is None 2024-04-25T16:27:03.992 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:27:04.219 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:27:04.244 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:27:04.244 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:27:04.244 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:27:04.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:27:04.380 DEBUG:teuthology.orchestra.run.smithi116:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:27:04.400 INFO:teuthology.orchestra.run.smithi116.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:27:04.400 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:27:04.400 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:27:04.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:27:04.641 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T16:27:04.667 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-3291.g994e2f20.el9 2024-04-25T16:27:04.667 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291.g994e2f20.el9 2024-04-25T16:27:04.667 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291.g994e2f20 is installed. 2024-04-25T16:27:04.669 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T16:27:04.669 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:27:04.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:27:04.700 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:27:04.701 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:27:04.727 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:27:04.727 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T16:27:04.757 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T16:27:04.758 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:27:04.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:27:04.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:27:04.852 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:27:04.852 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:27:04.875 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:27:04.938 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:27:04.938 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T16:27:04.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T16:27:05.033 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T16:27:05.034 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:27:05.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:27:05.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:27:05.140 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:27:05.140 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:27:05.163 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:27:05.226 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:27:05.226 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T16:27:05.254 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T16:27:05.319 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T16:27:05.320 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:27:05.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:27:05.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:27:05.406 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:27:05.406 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:27:05.430 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:27:05.493 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:27:05.493 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T16:27:05.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T16:27:05.588 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T16:27:05.685 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T16:27:05.686 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:27:05.686 INFO:tasks.cephadm:Cluster fsid is a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:27:05.686 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T16:27:05.687 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.112', 'mon.b': '172.21.15.116', 'mon.c': '172.21.15.133'} 2024-04-25T16:27:05.687 INFO:tasks.cephadm:First mon is mon.a on smithi112 2024-04-25T16:27:05.687 INFO:tasks.cephadm:First mgr is x 2024-04-25T16:27:05.687 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T16:27:05.687 DEBUG:teuthology.orchestra.run.smithi112:> sudo hostname $(hostname -s) 2024-04-25T16:27:05.711 DEBUG:teuthology.orchestra.run.smithi116:> sudo hostname $(hostname -s) 2024-04-25T16:27:05.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2024-04-25T16:27:05.762 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T16:27:05.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:27:05.909 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78872/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3291-g994e2f20', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3291.g994e2f20'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 04:02:18.990013', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240425.015853-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-25T16:27:06.192 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20240425.015853-debug, sha1 994e2f20102a20fa9a27966c14b625cc01884df8 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:27:06.194 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm 2024-04-25T16:27:06.194 DEBUG:teuthology.orchestra.run.smithi112:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:06.595 INFO:teuthology.orchestra.run.smithi112.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 16:27 /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:06.595 DEBUG:teuthology.orchestra.run.smithi116:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.048 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 16:27 /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.049 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.455 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-r--r--. 1 ubuntu ubuntu 781647 Apr 25 16:27 /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.455 DEBUG:teuthology.orchestra.run.smithi112:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.474 DEBUG:teuthology.orchestra.run.smithi116:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.491 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T16:27:07.528 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 on all hosts... 2024-04-25T16:27:07.528 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T16:27:07.531 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T16:27:07.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 pull 2024-04-25T16:27:08.984 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:27:09.065 INFO:teuthology.orchestra.run.smithi112.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:27:09.127 INFO:teuthology.orchestra.run.smithi116.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:27:41.458 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-04-25T16:27:41.459 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:27:41.459 INFO:teuthology.orchestra.run.smithi116.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T16:27:41.459 INFO:teuthology.orchestra.run.smithi116.stdout: "repo_digests": [ 2024-04-25T16:27:41.459 INFO:teuthology.orchestra.run.smithi116.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T16:27:41.459 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2024-04-25T16:27:41.459 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-04-25T16:27:50.955 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T16:27:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:27:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T16:27:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2024-04-25T16:27:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T16:27:50.956 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-04-25T16:27:50.956 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T16:28:02.951 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T16:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "image_id": "d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4", 2024-04-25T16:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "repo_digests": [ 2024-04-25T16:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385" 2024-04-25T16:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:28:02.952 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:28:02.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph 2024-04-25T16:28:03.022 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph 2024-04-25T16:28:03.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2024-04-25T16:28:03.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /etc/ceph 2024-04-25T16:28:03.109 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /etc/ceph 2024-04-25T16:28:03.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2024-04-25T16:28:03.164 INFO:tasks.cephadm:Writing seed config... 2024-04-25T16:28:03.165 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T16:28:03.165 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T16:28:03.165 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T16:28:03.165 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T16:28:03.165 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T16:28:03.166 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-25T16:28:03.167 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T16:28:03.168 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T16:28:03.169 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:28:03.169 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T16:28:03.184 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a7905982-0320-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T16:28:03.184 DEBUG:teuthology.orchestra.run.smithi112:mon.a> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T16:28:03.226 DEBUG:teuthology.orchestra.run.smithi112:mgr.x> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T16:28:03.269 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T16:28:03.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 -v bootstrap --fsid a7905982-0320-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.112 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:28:03.458 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2024-04-25T16:28:03.458 INFO:teuthology.orchestra.run.smithi112.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8', '-v', 'bootstrap', '--fsid', 'a7905982-0320-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.112', '--skip-admin-label'] 2024-04-25T16:28:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T16:28:03.459 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying podman|docker is present... 2024-04-25T16:28:03.484 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T16:28:03.484 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying lvm2 is present... 2024-04-25T16:28:03.484 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying time synchronization is in place... 2024-04-25T16:28:03.491 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T16:28:03.491 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T16:28:03.497 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T16:28:03.497 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T16:28:03.503 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-04-25T16:28:03.509 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-04-25T16:28:03.509 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-04-25T16:28:03.509 INFO:teuthology.orchestra.run.smithi112.stdout:Repeating the final host check... 2024-04-25T16:28:03.532 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi112.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl is present 2024-04-25T16:28:03.533 INFO:teuthology.orchestra.run.smithi112.stdout:lvcreate is present 2024-04-25T16:28:03.538 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T16:28:03.538 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T16:28:03.544 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T16:28:03.545 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T16:28:03.551 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout enabled 2024-04-25T16:28:03.557 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout active 2024-04-25T16:28:03.557 INFO:teuthology.orchestra.run.smithi112.stdout:Unit chronyd.service is enabled and running 2024-04-25T16:28:03.557 INFO:teuthology.orchestra.run.smithi112.stdout:Host looks OK 2024-04-25T16:28:03.557 INFO:teuthology.orchestra.run.smithi112.stdout:Cluster fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:03.557 INFO:teuthology.orchestra.run.smithi112.stdout:Acquiring lock 139677319511200 on /run/cephadm/a7905982-0320-11ef-bc93-c7b262605968.lock 2024-04-25T16:28:03.558 INFO:teuthology.orchestra.run.smithi112.stdout:Lock 139677319511200 acquired on /run/cephadm/a7905982-0320-11ef-bc93-c7b262605968.lock 2024-04-25T16:28:03.558 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 3300 ... 2024-04-25T16:28:03.558 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying IP 172.21.15.112 port 6789 ... 2024-04-25T16:28:03.558 INFO:teuthology.orchestra.run.smithi112.stdout:Base mon IP(s) is [172.21.15.112:3300, 172.21.15.112:6789], mon addrv is [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-04-25T16:28:03.561 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.112 metric 100 2024-04-25T16:28:03.561 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.112 metric 100 2024-04-25T16:28:03.563 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T16:28:03.564 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T16:28:03.564 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T16:28:03.566 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T16:28:03.566 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T16:28:03.566 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T16:28:03.566 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T16:28:03.566 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout inet6 fe80::7262:a08e:a16d:21d7/64 scope link noprefixroute 2024-04-25T16:28:03.566 INFO:teuthology.orchestra.run.smithi112.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T16:28:03.567 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-04-25T16:28:03.567 INFO:teuthology.orchestra.run.smithi112.stdout:Mon IP `172.21.15.112` is in CIDR network `172.21.0.0/20` 2024-04-25T16:28:03.567 INFO:teuthology.orchestra.run.smithi112.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T16:28:03.567 INFO:teuthology.orchestra.run.smithi112.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T16:28:03.568 INFO:teuthology.orchestra.run.smithi112.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stdout d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8... 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:665339d96aad80d083ba9988a62422bcef0a42210fb2b28340e33ab98589f7b8 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Copying config sha256:d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 2024-04-25T16:28:04.037 INFO:teuthology.orchestra.run.smithi112.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T16:28:04.982 INFO:teuthology.orchestra.run.smithi112.stdout:ceph: stdout ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T16:28:04.982 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph version: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev) 2024-04-25T16:28:04.982 INFO:teuthology.orchestra.run.smithi112.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T16:28:05.984 INFO:teuthology.orchestra.run.smithi112.stdout:stat: stdout 167 167 2024-04-25T16:28:05.984 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial keys... 2024-04-25T16:28:06.848 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCWhCpmlV3bExAAroscs93I+IUE5R5bySPZQA== 2024-04-25T16:28:07.762 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCXhCpmp6G7ChAAgl2JVADJWj1RlBkQ2yEbVw== 2024-04-25T16:28:08.760 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph-authtool: stdout AQCYhCpmqWC3ChAA+ELwtV7+e5b4gXOOJE4YYA== 2024-04-25T16:28:08.760 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial monmap... 2024-04-25T16:28:09.691 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool for a [v2:172.21.15.112:3300,v1:172.21.15.112:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: set fsid to a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:28:09.692 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mon... 2024-04-25T16:28:10.647 INFO:teuthology.orchestra.run.smithi112.stdout:create mon.a on 2024-04-25T16:28:10.840 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T16:28:10.978 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T16:28:11.141 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a7905982-0320-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-a7905982-0320-11ef-bc93-c7b262605968.target. 2024-04-25T16:28:11.141 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a7905982-0320-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-a7905982-0320-11ef-bc93-c7b262605968.target. 2024-04-25T16:28:11.371 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a 2024-04-25T16:28:11.371 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T16:28:11.535 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a7905982-0320-11ef-bc93-c7b262605968.target.wants/ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a7905982-0320-11ef-bc93-c7b262605968@.service. 2024-04-25T16:28:11.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:11 smithi112 podman[28725]: 2024-04-25 16:28:11.782421887 +0000 UTC m=+0.121718296 container create 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T16:28:12.099 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T16:28:12.099 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T16:28:12.106 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T16:28:12.107 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T16:28:12.107 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T16:28:12.107 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T16:28:12.107 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon to start... 2024-04-25T16:28:12.107 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mon... 2024-04-25T16:28:12.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:12 smithi112 podman[28725]: 2024-04-25 16:28:12.009542305 +0000 UTC m=+0.348838718 container init 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T16:28:12.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:12 smithi112 podman[28725]: 2024-04-25 16:28:12.012383324 +0000 UTC m=+0.351679741 container start 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T16:28:12.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:12 smithi112 ceph-mon[28744]: mkfs a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:12.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:12 smithi112 ceph-mon[28744]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T16:28:12.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:12 smithi112 bash[28725]: 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba 2024-04-25T16:28:12.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:12 smithi112 systemd[1]: Started Ceph mon.a for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:28:13.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T16:28:13.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: monmap epoch 1 2024-04-25T16:28:13.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:13.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: last_changed 2024-04-25T16:28:09.103195+0000 2024-04-25T16:28:13.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: min_mon_release 19 (squid) 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: election_strategy: 1 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: fsmap 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: mgrmap e1: no daemons active 2024-04-25T16:28:13.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:13 smithi112 ceph-mon[28744]: from='client.? 172.21.15.112:0/3833041389' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout id: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout services: 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.610993s) 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T16:28:13.234 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout data: 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.593+0000 7fc17c4db700 1 Processor -- start 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.593+0000 7fc17c4db700 1 -- start start 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.594+0000 7fc17c4db700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17410c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.594+0000 7fc17c4db700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410ccd0 con 0x7fc17407d560 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.594+0000 7fc17a277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17410c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.594+0000 7fc17a277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17410c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58658/0 (socket says 172.21.15.112:58658) 2024-04-25T16:28:13.235 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.594+0000 7fc17a277700 1 -- 172.21.15.112:0/277021072 learned_addr learned my addr 172.21.15.112:0/277021072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.594+0000 7fc17a277700 1 -- 172.21.15.112:0/277021072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17410ce50 con 0x7fc17407d560 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.595+0000 7fc17a277700 1 --2- 172.21.15.112:0/277021072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17410c6c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc1700098a0 tx=0x7fc170009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fba54c0c53d0e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.595+0000 7fc179275700 1 -- 172.21.15.112:0/277021072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc170004160 con 0x7fc17407d560 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.595+0000 7fc179275700 1 -- 172.21.15.112:0/277021072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc170004300 con 0x7fc17407d560 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.595+0000 7fc179275700 1 -- 172.21.15.112:0/277021072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1700036a0 con 0x7fc17407d560 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 -- 172.21.15.112:0/277021072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 msgr2=0x7fc17410c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 --2- 172.21.15.112:0/277021072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17410c6c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc1700098a0 tx=0x7fc170009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 -- 172.21.15.112:0/277021072 shutdown_connections 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 --2- 172.21.15.112:0/277021072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17410c6c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 -- 172.21.15.112:0/277021072 >> 172.21.15.112:0/277021072 conn(0x7fc174104150 msgr2=0x7fc1741065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 -- 172.21.15.112:0/277021072 shutdown_connections 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 -- 172.21.15.112:0/277021072 wait complete. 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 Processor -- start 2024-04-25T16:28:13.236 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.596+0000 7fc17c4db700 1 -- start start 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17c4db700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17c4db700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc17410d380 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17a277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17a277700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58660/0 (socket says 172.21.15.112:58660) 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17a277700 1 -- 172.21.15.112:0/3833041389 learned_addr learned my addr 172.21.15.112:0/3833041389 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17a277700 1 -- 172.21.15.112:0/3833041389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc17419dad0 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17a277700 1 --2- 172.21.15.112:0/3833041389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17419d590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc170000c00 tx=0x7fc170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc170003db0 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc170011630 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc170011950 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc17419dd60 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.597+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741a0970 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.598+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc170011af0 con 0x7fc17407d560 2024-04-25T16:28:13.237 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.598+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc17001c450 con 0x7fc17407d560 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.598+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc174064cf0 con 0x7fc17407d560 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.601+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc170032020 con 0x7fc17407d560 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.672+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc17402e810 con 0x7fc17407d560 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.673+0000 7fc16b7fe700 1 -- 172.21.15.112:0/3833041389 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc170004260 con 0x7fc17407d560 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 msgr2=0x7fc17419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 --2- 172.21.15.112:0/3833041389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17419d590 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc170000c00 tx=0x7fc170004750 comp rx=0 tx=0).stop 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 shutdown_connections 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 --2- 172.21.15.112:0/3833041389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc17407d560 0x7fc17419d590 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 >> 172.21.15.112:0/3833041389 conn(0x7fc174104150 msgr2=0x7fc174105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 shutdown_connections 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:12.674+0000 7fc17c4db700 1 -- 172.21.15.112:0/3833041389 wait complete. 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:mon is available 2024-04-25T16:28:13.238 INFO:teuthology.orchestra.run.smithi112.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.112:3300,v1:172.21.15.112:6789] 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T16:28:14.310 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f38694ab700 1 Processor -- start 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f38694ab700 1 -- start start 2024-04-25T16:28:14.311 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f38694ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f3864108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f38694ab700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3864109490 con 0x7f3864108a90 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f3862ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f3864108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f3862ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f3864108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58664/0 (socket says 172.21.15.112:58664) 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.698+0000 7f3862ffd700 1 -- 172.21.15.112:0/657777474 learned_addr learned my addr 172.21.15.112:0/657777474 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.699+0000 7f3862ffd700 1 -- 172.21.15.112:0/657777474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3864109bc0 con 0x7f3864108a90 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.699+0000 7f3862ffd700 1 --2- 172.21.15.112:0/657777474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f3864108ea0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f38540098a0 tx=0x7f3854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ab805c7243cca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.699+0000 7f3861ffb700 1 -- 172.21.15.112:0/657777474 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3854004160 con 0x7f3864108a90 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.699+0000 7f3861ffb700 1 -- 172.21.15.112:0/657777474 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3854004300 con 0x7f3864108a90 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.699+0000 7f3861ffb700 1 -- 172.21.15.112:0/657777474 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38540036a0 con 0x7f3864108a90 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.699+0000 7f38694ab700 1 -- 172.21.15.112:0/657777474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 msgr2=0x7f3864108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 --2- 172.21.15.112:0/657777474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f3864108ea0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f38540098a0 tx=0x7f3854009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 -- 172.21.15.112:0/657777474 shutdown_connections 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 --2- 172.21.15.112:0/657777474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f3864108ea0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:14.312 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 -- 172.21.15.112:0/657777474 >> 172.21.15.112:0/657777474 conn(0x7f3864104150 msgr2=0x7f38641065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 -- 172.21.15.112:0/657777474 shutdown_connections 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 -- 172.21.15.112:0/657777474 wait complete. 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 Processor -- start 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.700+0000 7f38694ab700 1 -- start start 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f38694ab700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f386419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f38694ab700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f386410a0f0 con 0x7f3864108a90 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f3862ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f386419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f3862ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f386419c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58678/0 (socket says 172.21.15.112:58678) 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f3862ffd700 1 -- 172.21.15.112:0/4000217761 learned_addr learned my addr 172.21.15.112:0/4000217761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f3862ffd700 1 -- 172.21.15.112:0/4000217761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386419c9b0 con 0x7f3864108a90 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f3862ffd700 1 --2- 172.21.15.112:0/4000217761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f386419c470 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3854000c00 tx=0x7f3854004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3854003db0 con 0x7f3864108a90 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3854011630 con 0x7f3864108a90 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386419cc40 con 0x7f3864108a90 2024-04-25T16:28:14.313 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3854011950 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.701+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386407c860 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.702+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3854011af0 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.702+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f385401c450 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.702+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f386419d0a0 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.705+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3854032020 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.771+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f3864064cf0 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.776+0000 7f384bfff700 1 -- 172.21.15.112:0/4000217761 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+467 (secure 0 0 0) 0x7f3854004260 con 0x7f3864108a90 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 msgr2=0x7f386419c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 --2- 172.21.15.112:0/4000217761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f386419c470 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f3854000c00 tx=0x7f3854004750 comp rx=0 tx=0).stop 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 shutdown_connections 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 --2- 172.21.15.112:0/4000217761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3864108a90 0x7f386419c470 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 >> 172.21.15.112:0/4000217761 conn(0x7f3864104150 msgr2=0x7f3864192b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 shutdown_connections 2024-04-25T16:28:14.314 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:13.777+0000 7f38694ab700 1 -- 172.21.15.112:0/4000217761 wait complete. 2024-04-25T16:28:14.315 INFO:teuthology.orchestra.run.smithi112.stdout:Generating new minimal ceph.conf... 2024-04-25T16:28:14.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:14 smithi112 ceph-mon[28744]: from='client.? 172.21.15.112:0/4000217761' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T16:28:14.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:14 smithi112 ceph-mon[28744]: from='client.? 172.21.15.112:0/4000217761' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T16:28:15.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.713+0000 7f97bbc4f700 1 Processor -- start 2024-04-25T16:28:15.316 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97bbc4f700 1 -- start start 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97bbc4f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b410cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97bbc4f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97b407d670 con 0x7f97b410a690 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97b99eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b410cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97b99eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b410cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58690/0 (socket says 172.21.15.112:58690) 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97b99eb700 1 -- 172.21.15.112:0/3106250115 learned_addr learned my addr 172.21.15.112:0/3106250115 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97b99eb700 1 -- 172.21.15.112:0/3106250115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b410cff0 con 0x7f97b410a690 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.714+0000 7f97b99eb700 1 --2- 172.21.15.112:0/3106250115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b410cab0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f97b00098a0 tx=0x7f97b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3689be28c020ae18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97b89e9700 1 -- 172.21.15.112:0/3106250115 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0004160 con 0x7f97b410a690 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97b89e9700 1 -- 172.21.15.112:0/3106250115 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f97b0004300 con 0x7f97b410a690 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97b89e9700 1 -- 172.21.15.112:0/3106250115 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0011af0 con 0x7f97b410a690 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97bbc4f700 1 -- 172.21.15.112:0/3106250115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 msgr2=0x7f97b410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:15.317 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97bbc4f700 1 --2- 172.21.15.112:0/3106250115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b410cab0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f97b00098a0 tx=0x7f97b0009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97bbc4f700 1 -- 172.21.15.112:0/3106250115 shutdown_connections 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97bbc4f700 1 --2- 172.21.15.112:0/3106250115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b410cab0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.715+0000 7f97bbc4f700 1 -- 172.21.15.112:0/3106250115 >> 172.21.15.112:0/3106250115 conn(0x7f97b4104130 msgr2=0x7f97b41065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.716+0000 7f97bbc4f700 1 -- 172.21.15.112:0/3106250115 shutdown_connections 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.716+0000 7f97bbc4f700 1 -- 172.21.15.112:0/3106250115 wait complete. 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.716+0000 7f97bbc4f700 1 Processor -- start 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.716+0000 7f97bbc4f700 1 -- start start 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.716+0000 7f97bbc4f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.716+0000 7f97bbc4f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f97b410def0 con 0x7f97b410a690 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97b99eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97b99eb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b419de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58698/0 (socket says 172.21.15.112:58698) 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97b99eb700 1 -- 172.21.15.112:0/1398603424 learned_addr learned my addr 172.21.15.112:0/1398603424 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:15.318 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97b99eb700 1 -- 172.21.15.112:0/1398603424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b419e370 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97b99eb700 1 --2- 172.21.15.112:0/1398603424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b419de30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009fb0 tx=0x7f97b0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b00036a0 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f97b001a430 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97b0019550 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b419e600 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a1210 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97b0019be0 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.717+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f97b0024730 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.718+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97b4064cf0 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.720+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f97b0043e50 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.787+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f97b41a13e0 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.787+0000 7f97aaffd700 1 -- 172.21.15.112:0/1398603424 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f97b0032020 con 0x7f97b410a690 2024-04-25T16:28:15.319 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 msgr2=0x7f97b419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 --2- 172.21.15.112:0/1398603424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b419de30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f97b0009fb0 tx=0x7f97b0011940 comp rx=0 tx=0).stop 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 shutdown_connections 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 --2- 172.21.15.112:0/1398603424 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97b410a690 0x7f97b419de30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 >> 172.21.15.112:0/1398603424 conn(0x7f97b4104130 msgr2=0x7f97b4104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 shutdown_connections 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:14.788+0000 7f97bbc4f700 1 -- 172.21.15.112:0/1398603424 wait complete. 2024-04-25T16:28:15.320 INFO:teuthology.orchestra.run.smithi112.stdout:Restarting the monitor... 2024-04-25T16:28:15.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:15 smithi112 ceph-mon[28744]: from='client.? 172.21.15.112:0/1398603424' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:28:15.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:15 smithi112 systemd[1]: Stopping Ceph mon.a for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:28:15.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a[28740]: 2024-04-25T16:28:15.570+0000 7f7071acc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:28:15.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a[28740]: 2024-04-25T16:28:15.570+0000 7f7071acc700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T16:28:16.036 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:15 smithi112 podman[29000]: 2024-04-25 16:28:15.755505792 +0000 UTC m=+0.278983672 container died 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T16:28:16.037 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 podman[29000]: 2024-04-25 16:28:16.037529581 +0000 UTC m=+0.561007451 container cleanup 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T16:28:16.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 bash[29000]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a 2024-04-25T16:28:16.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 podman[29013]: 2024-04-25 16:28:16.271546333 +0000 UTC m=+0.514762798 container remove 0567946d9403e65802ec814f7134d2d378597d58af8b27dbbbb1d956f8ed2fba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:28:16.710 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 systemd[1]: ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T16:28:16.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 systemd[1]: Stopped Ceph mon.a for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:28:16.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 systemd[1]: Starting Ceph mon.a for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:28:16.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 podman[29103]: 2024-04-25 16:28:16.606120356 +0000 UTC m=+0.104544648 container create 60a3b4a63620d25f1ecc2ae158fee36f7bc4c3fbe31fb8fa93f6e368ccc9c48b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:28:16.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 podman[29103]: 2024-04-25 16:28:16.519370532 +0000 UTC m=+0.017794830 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:28:16.895 INFO:teuthology.orchestra.run.smithi112.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T16:28:17.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 podman[29103]: 2024-04-25 16:28:16.808330653 +0000 UTC m=+0.306754944 container init 60a3b4a63620d25f1ecc2ae158fee36f7bc4c3fbe31fb8fa93f6e368ccc9c48b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 podman[29103]: 2024-04-25 16:28:16.811194093 +0000 UTC m=+0.309618385 container start 60a3b4a63620d25f1ecc2ae158fee36f7bc4c3fbe31fb8fa93f6e368ccc9c48b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: pidfile_write: ignore empty --pid-file 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: load: jerasure load: lrc 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: RocksDB version: 7.9.2 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Git sha 0 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: DB SUMMARY 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: DB Session ID: 6AZA0OR1XRMOM3HR1SF8 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: CURRENT file: CURRENT 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T16:28:17.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105710 ; 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.error_if_exists: 0 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.create_if_missing: 0 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.paranoid_checks: 1 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.env: 0x56237410f500 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.info_log: 0x5623766b7fc0 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.statistics: (nil) 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.use_fsync: 0 2024-04-25T16:28:17.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_log_file_size: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.allow_fallocate: 1 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.use_direct_reads: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.db_log_dir: 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.wal_dir: 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T16:28:17.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.write_buffer_manager: 0x5623766888c0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T16:28:17.101 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.unordered_write: 0 2024-04-25T16:28:17.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T16:28:17.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T16:28:17.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T16:28:17.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T16:28:17.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.row_cache: None 2024-04-25T16:28:17.102 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.wal_filter: None 2024-04-25T16:28:17.103 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T16:28:17.103 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T16:28:17.103 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.two_write_queues: 0 2024-04-25T16:28:17.103 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.wal_compression: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.atomic_flush: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.log_readahead_size: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_background_jobs: 2 2024-04-25T16:28:17.104 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_background_compactions: -1 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_subcompactions: 1 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_open_files: -1 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T16:28:17.105 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_background_flushes: -1 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Compression algorithms supported: 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kZSTD supported: 0 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kXpressCompression supported: 0 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kZlibCompression supported: 1 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kSnappyCompression supported: 1 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kLZ4Compression supported: 1 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: kBZip2Compression supported: 0 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T16:28:17.106 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.merge_operator: 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_filter: None 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5623767be1a0) 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks: 1 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: pin_top_level_index_and_filter: 1 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_type: 0 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_index_type: 0 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_shortening: 1 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T16:28:17.107 INFO:journalctl@ceph.mon.a.smithi112.stdout: checksum: 4 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: no_block_cache: 0 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache: 0x5623766bb1f0 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_name: BinnedLRUCache 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_options: 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: capacity : 536870912 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_shard_bits : 4 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: strict_capacity_limit : 0 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: high_pri_pool_ratio: 0.000 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_cache_compressed: (nil) 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: persistent_cache: (nil) 2024-04-25T16:28:17.108 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size: 4096 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_size_deviation: 10 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_restart_interval: 16 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: index_block_restart_interval: 1 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: metadata_block_size: 4096 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: partition_filters: 0 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: use_delta_encoding: 1 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: filter_policy: bloomfilter 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: whole_key_filtering: 1 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: verify_compression: 0 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: read_amp_bytes_per_bit: 0 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: format_version: 5 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_index_compression: 1 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: block_align: 0 2024-04-25T16:28:17.109 INFO:journalctl@ceph.mon.a.smithi112.stdout: max_auto_readahead_size: 262144 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout: prepopulate_block_cache: 0 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout: initial_auto_readahead_size: 8192 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression: NoCompression 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.num_levels: 7 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T16:28:17.110 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T16:28:17.111 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T16:28:17.112 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T16:28:17.113 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.inplace_update_support: 0 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.bloom_locality: 0 2024-04-25T16:28:17.114 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.max_successive_merges: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.ttl: 2592000 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T16:28:17.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.enable_blob_files: false 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.min_blob_size: 0 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T16:28:17.116 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 78a673c4-3de8-4cc6-94b5-0250c27e8f1c 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714062496850622, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T16:28:17.117 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714062496851713, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100817, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98705, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90923, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714062496, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "78a673c4-3de8-4cc6-94b5-0250c27e8f1c", "db_session_id": "6AZA0OR1XRMOM3HR1SF8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714062496851795, "job": 1, "event": "recovery_finished"} 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56237669fc00 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: DB pointer 0x56237b492000 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** DB Stats ** 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T16:28:17.118 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: L0 2/0 100.31 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Sum 2/0 100.31 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** Compaction Stats [default] ** 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T16:28:17.119 INFO:journalctl@ceph.mon.a.smithi112.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Cumulative compaction: 0.00 GB write, 30.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Interval compaction: 0.00 GB write, 30.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T16:28:17.120 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache BinnedLRUCache@0x5623766bb1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: starting mon.a rank 0 at public addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] at bind addrs [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???) e1 preinit fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).mds e1 new map 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).mds e1 print_map 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: e1 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: legacy client fscid: -1 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout: No filesystems configured 2024-04-25T16:28:17.121 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 bash[29103]: 60a3b4a63620d25f1ecc2ae158fee36f7bc4c3fbe31fb8fa93f6e368ccc9c48b 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 systemd[1]: Started Ceph mon.a for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: monmap epoch 1 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: last_changed 2024-04-25T16:28:09.103195+0000 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: min_mon_release 19 (squid) 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: election_strategy: 1 2024-04-25T16:28:17.122 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:28:17.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: fsmap 2024-04-25T16:28:17.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T16:28:17.123 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:16 smithi112 ceph-mon[29122]: mgrmap e1: no daemons active 2024-04-25T16:28:17.883 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.298+0000 7f48e8c34700 1 Processor -- start 2024-04-25T16:28:17.887 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.298+0000 7f48e8c34700 1 -- start start 2024-04-25T16:28:17.887 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.298+0000 7f48e8c34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e4108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:17.887 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.298+0000 7f48e8c34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48e4109490 con 0x7f48e4108a90 2024-04-25T16:28:17.887 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.299+0000 7f48e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e4108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.299+0000 7f48e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e4108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57598/0 (socket says 172.21.15.112:57598) 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.299+0000 7f48e259c700 1 -- 172.21.15.112:0/3835451584 learned_addr learned my addr 172.21.15.112:0/3835451584 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.299+0000 7f48e259c700 1 -- 172.21.15.112:0/3835451584 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e4109bc0 con 0x7f48e4108a90 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.299+0000 7f48e259c700 1 --2- 172.21.15.112:0/3835451584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e4108ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f48d40098a0 tx=0x7f48d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886203be9441e801 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.299+0000 7f48e1d9b700 1 -- 172.21.15.112:0/3835451584 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d4004160 con 0x7f48e4108a90 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e1d9b700 1 -- 172.21.15.112:0/3835451584 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f48d4004300 con 0x7f48e4108a90 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e1d9b700 1 -- 172.21.15.112:0/3835451584 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d4011af0 con 0x7f48e4108a90 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 -- 172.21.15.112:0/3835451584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 msgr2=0x7f48e4108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 --2- 172.21.15.112:0/3835451584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e4108ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f48d40098a0 tx=0x7f48d4009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 -- 172.21.15.112:0/3835451584 shutdown_connections 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 --2- 172.21.15.112:0/3835451584 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e4108ea0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 -- 172.21.15.112:0/3835451584 >> 172.21.15.112:0/3835451584 conn(0x7f48e4104150 msgr2=0x7f48e41065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 -- 172.21.15.112:0/3835451584 shutdown_connections 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.300+0000 7f48e8c34700 1 -- 172.21.15.112:0/3835451584 wait complete. 2024-04-25T16:28:17.888 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.301+0000 7f48e8c34700 1 Processor -- start 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.301+0000 7f48e8c34700 1 -- start start 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.301+0000 7f48e8c34700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.301+0000 7f48e8c34700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48e410cf70 con 0x7f48e4108a90 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e259c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e419def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57614/0 (socket says 172.21.15.112:57614) 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e259c700 1 -- 172.21.15.112:0/1197631705 learned_addr learned my addr 172.21.15.112:0/1197631705 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e259c700 1 -- 172.21.15.112:0/1197631705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e419e430 con 0x7f48e4108a90 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e259c700 1 --2- 172.21.15.112:0/1197631705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e419def0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009fb0 tx=0x7f48d4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d40036a0 con 0x7f48e4108a90 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e419e6c0 con 0x7f48e4108a90 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f48d4003cd0 con 0x7f48e4108a90 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48d401a430 con 0x7f48e4108a90 2024-04-25T16:28:17.889 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e41a12d0 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.302+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f48d401aa80 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.303+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f48d40193c0 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.303+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48e4064cf0 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.306+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f48d4024440 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.373+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f48e402e510 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.376+0000 7f48dbfff700 1 -- 172.21.15.112:0/1197631705 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f48d4024760 con 0x7f48e4108a90 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.377+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 msgr2=0x7f48e419def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.377+0000 7f48e8c34700 1 --2- 172.21.15.112:0/1197631705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e419def0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f48d4009fb0 tx=0x7f48d4011890 comp rx=0 tx=0).stop 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.377+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 shutdown_connections 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.377+0000 7f48e8c34700 1 --2- 172.21.15.112:0/1197631705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48e4108a90 0x7f48e419def0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.377+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 >> 172.21.15.112:0/1197631705 conn(0x7f48e4104150 msgr2=0x7f48e4194480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.378+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 shutdown_connections 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:17.378+0000 7f48e8c34700 1 -- 172.21.15.112:0/1197631705 wait complete. 2024-04-25T16:28:17.890 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T16:28:17.905 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:28:17.905 INFO:teuthology.orchestra.run.smithi112.stdout:Creating mgr... 2024-04-25T16:28:17.905 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T16:28:17.905 INFO:teuthology.orchestra.run.smithi112.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T16:28:18.151 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x 2024-04-25T16:28:18.151 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T16:28:18.296 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a7905982-0320-11ef-bc93-c7b262605968.target.wants/ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a7905982-0320-11ef-bc93-c7b262605968@.service. 2024-04-25T16:28:18.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:18 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1197631705' entity='client.admin' 2024-04-25T16:28:18.597 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:18 smithi112 podman[29336]: 2024-04-25 16:28:18.437672749 +0000 UTC m=+0.017698292 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:28:18.841 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T16:28:18.841 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T16:28:18.848 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T16:28:18.848 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T16:28:18.848 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T16:28:18.848 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T16:28:18.856 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T16:28:18.856 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T16:28:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T16:28:18.863 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T16:28:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T16:28:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T16:28:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr to start... 2024-04-25T16:28:18.864 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr... 2024-04-25T16:28:18.928 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:18 smithi112 podman[29336]: 2024-04-25 16:28:18.759938199 +0000 UTC m=+0.339963738 container init d8c4e38aecbcb3ee5e0c99e4c1ee0acb6c71d5f44bf19ae5044ea5ea2c47cad2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T16:28:18.928 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:18 smithi112 podman[29336]: 2024-04-25 16:28:18.76243194 +0000 UTC m=+0.342457488 container start d8c4e38aecbcb3ee5e0c99e4c1ee0acb6c71d5f44bf19ae5044ea5ea2c47cad2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T16:28:18.929 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:18 smithi112 bash[29336]: d8c4e38aecbcb3ee5e0c99e4c1ee0acb6c71d5f44bf19ae5044ea5ea2c47cad2 2024-04-25T16:28:18.929 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:18 smithi112 systemd[1]: Started Ceph mgr.x for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:28:19.220 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:18.928+0000 7fe212566200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:28:19.220 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:19 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:19.117+0000 7fe212566200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:28:19.220 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:19 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:19.219+0000 7fe212566200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:28:19.772 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:19 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/184052062' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T16:28:19.773 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:19 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:19.559+0000 7fe212566200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:28:19.773 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:19 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:19.644+0000 7fe212566200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T16:28:20.016 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T16:28:20.017 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:20.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T16:28:20.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T16:28:20.018 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T16:28:20.025 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T16:28:20.026 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:20.027 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T16:28:12.064506+0000", 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.342+0000 7f6df6cd1700 1 Processor -- start 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.342+0000 7f6df6cd1700 1 -- start start 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.342+0000 7f6df6cd1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df0108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.342+0000 7f6df6cd1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6df0109420 con 0x7f6df0108a20 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.343+0000 7f6df4a6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df0108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.343+0000 7f6df4a6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df0108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57642/0 (socket says 172.21.15.112:57642) 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.343+0000 7f6df4a6d700 1 -- 172.21.15.112:0/3064283828 learned_addr learned my addr 172.21.15.112:0/3064283828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:20.028 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.343+0000 7f6df4a6d700 1 -- 172.21.15.112:0/3064283828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df0109b50 con 0x7f6df0108a20 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.343+0000 7f6df4a6d700 1 --2- 172.21.15.112:0/3064283828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df0108e30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098a0 tx=0x7f6de4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29019dc3ae6c820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.343+0000 7f6def7fe700 1 -- 172.21.15.112:0/3064283828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de4004160 con 0x7f6df0108a20 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6def7fe700 1 -- 172.21.15.112:0/3064283828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6de4004300 con 0x7f6df0108a20 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6def7fe700 1 -- 172.21.15.112:0/3064283828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de4011af0 con 0x7f6df0108a20 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 -- 172.21.15.112:0/3064283828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 msgr2=0x7f6df0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 --2- 172.21.15.112:0/3064283828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df0108e30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6de40098a0 tx=0x7f6de4009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 -- 172.21.15.112:0/3064283828 shutdown_connections 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 --2- 172.21.15.112:0/3064283828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df0108e30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 -- 172.21.15.112:0/3064283828 >> 172.21.15.112:0/3064283828 conn(0x7f6df01040e0 msgr2=0x7f6df0106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 -- 172.21.15.112:0/3064283828 shutdown_connections 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.344+0000 7f6df6cd1700 1 -- 172.21.15.112:0/3064283828 wait complete. 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.345+0000 7f6df6cd1700 1 Processor -- start 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.345+0000 7f6df6cd1700 1 -- start start 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.345+0000 7f6df6cd1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df01071b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:20.029 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.345+0000 7f6df6cd1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6df010cf00 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.345+0000 7f6df4a6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df01071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6df4a6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df01071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57654/0 (socket says 172.21.15.112:57654) 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6df4a6d700 1 -- 172.21.15.112:0/184052062 learned_addr learned my addr 172.21.15.112:0/184052062 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6df4a6d700 1 -- 172.21.15.112:0/184052062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df01076f0 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6df4a6d700 1 --2- 172.21.15.112:0/184052062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df01071b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009fb0 tx=0x7f6de4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de40036a0 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6de4003cd0 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df0105940 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6de401a430 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.346+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df0105e20 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.347+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6de401aa80 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.347+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6de40193c0 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.348+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df0064cf0 con 0x7f6df0108a20 2024-04-25T16:28:20.030 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.350+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6de4024440 con 0x7f6df0108a20 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.435+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6df0196fb0 con 0x7f6df0108a20 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.435+0000 7f6dedffb700 1 -- 172.21.15.112:0/184052062 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6de4024760 con 0x7f6df0108a20 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 msgr2=0x7f6df01071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 --2- 172.21.15.112:0/184052062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df01071b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009fb0 tx=0x7f6de4011890 comp rx=0 tx=0).stop 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 shutdown_connections 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 --2- 172.21.15.112:0/184052062 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6df0108a20 0x7f6df01071b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 >> 172.21.15.112:0/184052062 conn(0x7f6df01040e0 msgr2=0x7f6df0194410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 shutdown_connections 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:19.437+0000 7f6df6cd1700 1 -- 172.21.15.112:0/184052062 wait complete. 2024-04-25T16:28:20.031 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (1/15)... 2024-04-25T16:28:21.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:20 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:20.840+0000 7fe212566200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:28:21.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:21.007+0000 7fe212566200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:28:21.459 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:21.176+0000 7fe212566200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:28:21.459 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:21.458+0000 7fe212566200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:28:21.749 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:21.549+0000 7fe212566200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:28:21.749 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:21.749+0000 7fe212566200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:28:22.098 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:21.938+0000 7fe212566200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:28:22.782 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1449028333' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T16:28:22.782 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:22 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:22.573+0000 7fe212566200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:28:23.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:23.005+0000 7fe212566200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:28:23.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:23.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:28:23.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:28:23.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T16:28:23.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T16:28:23.207 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.208 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T16:28:23.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:23.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T16:28:23.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T16:28:23.209 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T16:28:23.217 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T16:28:23.218 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T16:28:12.064506+0000", 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.423+0000 7fbf15cbb700 1 Processor -- start 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.423+0000 7fbf15cbb700 1 -- start start 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.423+0000 7fbf15cbb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1010c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf15cbb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010cc60 con 0x7fbf1007ccd0 2024-04-25T16:28:23.219 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1010c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1010c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57668/0 (socket says 172.21.15.112:57668) 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0f7fe700 1 -- 172.21.15.112:0/429491366 learned_addr learned my addr 172.21.15.112:0/429491366 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0f7fe700 1 -- 172.21.15.112:0/429491366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1010cde0 con 0x7fbf1007ccd0 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0f7fe700 1 --2- 172.21.15.112:0/429491366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1010c650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf000098a0 tx=0x7fbf00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9946a8de76d91495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0e7fc700 1 -- 172.21.15.112:0/429491366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf00004160 con 0x7fbf1007ccd0 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0e7fc700 1 -- 172.21.15.112:0/429491366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbf00004300 con 0x7fbf1007ccd0 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.424+0000 7fbf0e7fc700 1 -- 172.21.15.112:0/429491366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf00011af0 con 0x7fbf1007ccd0 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 -- 172.21.15.112:0/429491366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 msgr2=0x7fbf1010c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 --2- 172.21.15.112:0/429491366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1010c650 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf000098a0 tx=0x7fbf00009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 -- 172.21.15.112:0/429491366 shutdown_connections 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 --2- 172.21.15.112:0/429491366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1010c650 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 -- 172.21.15.112:0/429491366 >> 172.21.15.112:0/429491366 conn(0x7fbf101040e0 msgr2=0x7fbf10106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 -- 172.21.15.112:0/429491366 shutdown_connections 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.425+0000 7fbf15cbb700 1 -- 172.21.15.112:0/429491366 wait complete. 2024-04-25T16:28:23.220 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.426+0000 7fbf15cbb700 1 Processor -- start 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.426+0000 7fbf15cbb700 1 -- start start 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf15cbb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf15cbb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1010de60 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf0f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf0f7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57676/0 (socket says 172.21.15.112:57676) 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf0f7fe700 1 -- 172.21.15.112:0/1449028333 learned_addr learned my addr 172.21.15.112:0/1449028333 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf0f7fe700 1 -- 172.21.15.112:0/1449028333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1019e360 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf0f7fe700 1 --2- 172.21.15.112:0/1449028333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1019de20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009fb0 tx=0x7fbf00011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.427+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf000036a0 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.428+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbf0001a430 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.428+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1019e5f0 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.428+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf00019550 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.428+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf101a1200 con 0x7fbf1007ccd0 2024-04-25T16:28:23.221 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.428+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbf00019b90 con 0x7fbf1007ccd0 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.428+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbf00024730 con 0x7fbf1007ccd0 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.429+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf10064cf0 con 0x7fbf1007ccd0 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.431+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbf00043e50 con 0x7fbf1007ccd0 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.504+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbf101a13d0 con 0x7fbf1007ccd0 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.504+0000 7fbf0cff9700 1 -- 172.21.15.112:0/1449028333 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fbf00032020 con 0x7fbf1007ccd0 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 msgr2=0x7fbf1019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 --2- 172.21.15.112:0/1449028333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1019de20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009fb0 tx=0x7fbf00011940 comp rx=0 tx=0).stop 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 shutdown_connections 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 --2- 172.21.15.112:0/1449028333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbf1007ccd0 0x7fbf1019de20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 >> 172.21.15.112:0/1449028333 conn(0x7fbf101040e0 msgr2=0x7fbf10105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 shutdown_connections 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:22.505+0000 7fbf15cbb700 1 -- 172.21.15.112:0/1449028333 wait complete. 2024-04-25T16:28:23.222 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (2/15)... 2024-04-25T16:28:23.355 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:23.100+0000 7fe212566200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:28:23.355 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:23.303+0000 7fe212566200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:28:24.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:23.827+0000 7fe212566200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:28:24.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:23.919+0000 7fe212566200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:28:24.502 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:24.160+0000 7fe212566200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:28:24.502 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:24.292+0000 7fe212566200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:28:24.502 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:24.386+0000 7fe212566200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:28:24.502 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:24.501+0000 7fe212566200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:28:25.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:24.804+0000 7fe212566200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:28:25.597 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:25.411+0000 7fe212566200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:28:26.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:25.636+0000 7fe212566200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:28:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: Activating manager daemon x 2024-04-25T16:28:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: mgrmap e2: x(active, starting, since 0.00370281s) 2024-04-25T16:28:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T16:28:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T16:28:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: Manager daemon x is now available 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' 2024-04-25T16:28:26.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:25 smithi112 ceph-mon[29122]: from='mgr.14100 172.21.15.112:0/3085234274' entity='mgr.x' 2024-04-25T16:28:26.338 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T16:28:26.339 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T16:28:26.340 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T16:28:26.341 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T16:28:26.342 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T16:28:12.064506+0000", 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:28:26.343 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.639+0000 7f662f8e8700 1 Processor -- start 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.639+0000 7f662f8e8700 1 -- start start 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.639+0000 7f662f8e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628108e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.639+0000 7f662f8e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6628109470 con 0x7f6628108a70 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.639+0000 7f662d684700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628108e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.640+0000 7f662d684700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628108e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39282/0 (socket says 172.21.15.112:39282) 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.640+0000 7f662d684700 1 -- 172.21.15.112:0/2388733673 learned_addr learned my addr 172.21.15.112:0/2388733673 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.640+0000 7f662d684700 1 -- 172.21.15.112:0/2388733673 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6628109ba0 con 0x7f6628108a70 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.640+0000 7f662d684700 1 --2- 172.21.15.112:0/2388733673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628108e80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f66240098a0 tx=0x7f6624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a854777961906f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.642+0000 7f661ffff700 1 -- 172.21.15.112:0/2388733673 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6624004160 con 0x7f6628108a70 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.642+0000 7f661ffff700 1 -- 172.21.15.112:0/2388733673 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6624004300 con 0x7f6628108a70 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.642+0000 7f661ffff700 1 -- 172.21.15.112:0/2388733673 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6624011af0 con 0x7f6628108a70 2024-04-25T16:28:26.344 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 -- 172.21.15.112:0/2388733673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 msgr2=0x7f6628108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 --2- 172.21.15.112:0/2388733673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628108e80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f66240098a0 tx=0x7f6624009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 -- 172.21.15.112:0/2388733673 shutdown_connections 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 --2- 172.21.15.112:0/2388733673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628108e80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 -- 172.21.15.112:0/2388733673 >> 172.21.15.112:0/2388733673 conn(0x7f6628104130 msgr2=0x7f6628106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 -- 172.21.15.112:0/2388733673 shutdown_connections 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.643+0000 7f662f8e8700 1 -- 172.21.15.112:0/2388733673 wait complete. 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662f8e8700 1 Processor -- start 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662f8e8700 1 -- start start 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662f8e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662f8e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f662810cf50 con 0x7f6628108a70 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662d684700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662d684700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628107220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39300/0 (socket says 172.21.15.112:39300) 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.644+0000 7f662d684700 1 -- 172.21.15.112:0/244983860 learned_addr learned my addr 172.21.15.112:0/244983860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:26.345 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f662d684700 1 -- 172.21.15.112:0/244983860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6628107760 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f662d684700 1 --2- 172.21.15.112:0/244983860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628107220 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6624011890 tx=0x7f6624011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66240039e0 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6624003b80 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66240246e0 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66281059b0 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.645+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6628105e90 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.646+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6628064cf0 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.646+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 2) ==== 49272+0+0 (secure 0 0 0) 0x7f6624022020 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.647+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6624051f40 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.649+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6624005a20 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.724+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6628197130 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.724+0000 7f661e7fc700 1 -- 172.21.15.112:0/244983860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6624024970 con 0x7f6628108a70 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 msgr2=0x7f6628107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:26.346 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 --2- 172.21.15.112:0/244983860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628107220 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6624011890 tx=0x7f6624011970 comp rx=0 tx=0).stop 2024-04-25T16:28:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 shutdown_connections 2024-04-25T16:28:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 --2- 172.21.15.112:0/244983860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6628108a70 0x7f6628107220 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 >> 172.21.15.112:0/244983860 conn(0x7f6628104130 msgr2=0x7f6628194480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 shutdown_connections 2024-04-25T16:28:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:25.725+0000 7f662f8e8700 1 -- 172.21.15.112:0/244983860 wait complete. 2024-04-25T16:28:26.347 INFO:teuthology.orchestra.run.smithi112.stdout:mgr not available, waiting (3/15)... 2024-04-25T16:28:27.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:26 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/244983860' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T16:28:27.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:26 smithi112 ceph-mon[29122]: mgrmap e3: x(active, since 1.00889s) 2024-04-25T16:28:28.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:28 smithi112 ceph-mon[29122]: mgrmap e4: x(active, since 2s) 2024-04-25T16:28:32.115 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:31 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2048280909' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T16:28:38.574 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:38.574 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:28:38.574 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 0 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-04-25T16:28:38.575 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T16:28:38.576 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T16:28:38.585 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T16:28:38.586 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ], 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T16:28:12.064506+0000", 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout }, 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.565+0000 7f069658f700 1 Processor -- start 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.565+0000 7f069658f700 1 -- start start 2024-04-25T16:28:38.587 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f069658f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f0690109fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f069658f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f069010a5b0 con 0x7f0690109bb0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f0690109fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f0690109fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39352/0 (socket says 172.21.15.112:39352) 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068ffff700 1 -- 172.21.15.112:0/1927591107 learned_addr learned my addr 172.21.15.112:0/1927591107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068ffff700 1 -- 172.21.15.112:0/1927591107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069010ac90 con 0x7f0690109bb0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068ffff700 1 --2- 172.21.15.112:0/1927591107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f0690109fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06800098a0 tx=0x7f0680009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ac4dc9c39fa3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068effd700 1 -- 172.21.15.112:0/1927591107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0680004160 con 0x7f0690109bb0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068effd700 1 -- 172.21.15.112:0/1927591107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0680004300 con 0x7f0690109bb0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.566+0000 7f068effd700 1 -- 172.21.15.112:0/1927591107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0680011af0 con 0x7f0690109bb0 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 -- 172.21.15.112:0/1927591107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 msgr2=0x7f0690109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 --2- 172.21.15.112:0/1927591107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f0690109fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06800098a0 tx=0x7f0680009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 -- 172.21.15.112:0/1927591107 shutdown_connections 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 --2- 172.21.15.112:0/1927591107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f0690109fc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:38.588 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 -- 172.21.15.112:0/1927591107 >> 172.21.15.112:0/1927591107 conn(0x7f0690105250 msgr2=0x7f06901076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 -- 172.21.15.112:0/1927591107 shutdown_connections 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.567+0000 7f069658f700 1 -- 172.21.15.112:0/1927591107 wait complete. 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f069658f700 1 Processor -- start 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f069658f700 1 -- start start 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f069658f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f069019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f069658f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f069010e040 con 0x7f0690109bb0 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f068ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f069019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f068ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f069019dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39362/0 (socket says 172.21.15.112:39362) 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.568+0000 7f068ffff700 1 -- 172.21.15.112:0/2048280909 learned_addr learned my addr 172.21.15.112:0/2048280909 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f068ffff700 1 -- 172.21.15.112:0/2048280909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069019e3e0 con 0x7f0690109bb0 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f068ffff700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f069019dea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0680009fb0 tx=0x7f0680011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06800036a0 con 0x7f0690109bb0 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0680003cd0 con 0x7f0690109bb0 2024-04-25T16:28:38.589 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f068001a430 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069019e670 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.569+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06901a1280 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.570+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0690064cf0 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.570+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f068001a6a0 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.570+0000 7f068d7fa700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f067803d5e0 0x7f067803faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.570+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0680052410 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.570+0000 7f068f7fe700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f067803d5e0 0x7f067803faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.571+0000 7f068f7fe700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f067803d5e0 0x7f067803faf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0684006fd0 tx=0x7f0684006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.576+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f068001aa40 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.859+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f06901a1900 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.859+0000 7f068d7fa700 1 -- 172.21.15.112:0/2048280909 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0680020240 con 0x7f0690109bb0 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f067803d5e0 msgr2=0x7f067803faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f067803d5e0 0x7f067803faf0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0684006fd0 tx=0x7f0684006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:38.590 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 msgr2=0x7f069019dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f069019dea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0680009fb0 tx=0x7f0680011890 comp rx=0 tx=0).stop 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 shutdown_connections 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f067803d5e0 0x7f067803faf0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 --2- 172.21.15.112:0/2048280909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0690109bb0 0x7f069019dea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 >> 172.21.15.112:0/2048280909 conn(0x7f0690105250 msgr2=0x7f0690105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 shutdown_connections 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:31.861+0000 7f069658f700 1 -- 172.21.15.112:0/2048280909 wait complete. 2024-04-25T16:28:38.591 INFO:teuthology.orchestra.run.smithi112.stdout:mgr is available 2024-04-25T16:28:40.327 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:39 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3027154914' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T16:28:40.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:40.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T16:28:40.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout fsid = a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:40.464 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T16:28:40.465 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde42520700 1 Processor -- start 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde42520700 1 -- start start 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde42520700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c108e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde42520700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c109470 con 0x7fde3c108a70 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c108e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c108e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51822/0 (socket says 172.21.15.112:51822) 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde3bfff700 1 -- 172.21.15.112:0/601015040 learned_addr learned my addr 172.21.15.112:0/601015040 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.609+0000 7fde3bfff700 1 -- 172.21.15.112:0/601015040 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c109ba0 con 0x7fde3c108a70 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde3bfff700 1 --2- 172.21.15.112:0/601015040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c108e80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0098a0 tx=0x7fde2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a0eec878b5fa5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde3affd700 1 -- 172.21.15.112:0/601015040 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde2c004160 con 0x7fde3c108a70 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde3affd700 1 -- 172.21.15.112:0/601015040 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde2c004300 con 0x7fde3c108a70 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde3affd700 1 -- 172.21.15.112:0/601015040 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde2c011af0 con 0x7fde3c108a70 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde42520700 1 -- 172.21.15.112:0/601015040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 msgr2=0x7fde3c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde42520700 1 --2- 172.21.15.112:0/601015040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c108e80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0098a0 tx=0x7fde2c009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:40.466 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde42520700 1 -- 172.21.15.112:0/601015040 shutdown_connections 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde42520700 1 --2- 172.21.15.112:0/601015040 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c108e80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.610+0000 7fde42520700 1 -- 172.21.15.112:0/601015040 >> 172.21.15.112:0/601015040 conn(0x7fde3c1040f0 msgr2=0x7fde3c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.611+0000 7fde42520700 1 -- 172.21.15.112:0/601015040 shutdown_connections 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.611+0000 7fde42520700 1 -- 172.21.15.112:0/601015040 wait complete. 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.611+0000 7fde42520700 1 Processor -- start 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.611+0000 7fde42520700 1 -- start start 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde42520700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c19de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde42520700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c10cf50 con 0x7fde3c108a70 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c19de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c19de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51826/0 (socket says 172.21.15.112:51826) 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde3bfff700 1 -- 172.21.15.112:0/3027154914 learned_addr learned my addr 172.21.15.112:0/3027154914 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde3bfff700 1 -- 172.21.15.112:0/3027154914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c19e3c0 con 0x7fde3c108a70 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde3bfff700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c19de80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009fb0 tx=0x7fde2c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:40.467 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde2c0036a0 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fde2c003cd0 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c19e650 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde2c01a430 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.612+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c1a1260 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.613+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7fde2c01a6f0 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.613+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3c064cf0 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.613+0000 7fde397fa700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fde2803d590 0x7fde2803faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.613+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde2c052480 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.613+0000 7fde3b7fe700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fde2803d590 0x7fde2803faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.614+0000 7fde3b7fe700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fde2803d590 0x7fde2803faa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde24006fd0 tx=0x7fde24006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.619+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde2c017e90 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.814+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fde3c02e770 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.815+0000 7fde397fa700 1 -- 172.21.15.112:0/3027154914 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fde2c02c350 con 0x7fde3c108a70 2024-04-25T16:28:40.468 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.817+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fde2803d590 msgr2=0x7fde2803faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.817+0000 7fde42520700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fde2803d590 0x7fde2803faa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde24006fd0 tx=0x7fde24006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.817+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 msgr2=0x7fde3c19de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.817+0000 7fde42520700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c19de80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009fb0 tx=0x7fde2c011890 comp rx=0 tx=0).stop 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.818+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 shutdown_connections 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.818+0000 7fde42520700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fde2803d590 0x7fde2803faa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.818+0000 7fde42520700 1 --2- 172.21.15.112:0/3027154914 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde3c108a70 0x7fde3c19de80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.818+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 >> 172.21.15.112:0/3027154914 conn(0x7fde3c1040f0 msgr2=0x7fde3c1943b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.818+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 shutdown_connections 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:39.818+0000 7fde42520700 1 -- 172.21.15.112:0/3027154914 wait complete. 2024-04-25T16:28:40.469 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling cephadm module... 2024-04-25T16:28:41.365 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1728158071' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T16:28:41.366 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: ignoring --setuser ceph since I am not root 2024-04-25T16:28:41.366 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: ignoring --setgroup ceph since I am not root 2024-04-25T16:28:41.366 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:41.365+0000 7f33863fa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:28:41.659 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:41.555+0000 7f33863fa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:28:41.659 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:41.659+0000 7f33863fa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:28:41.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.851+0000 7f228f8b2700 1 Processor -- start 2024-04-25T16:28:41.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.851+0000 7f228f8b2700 1 -- start start 2024-04-25T16:28:41.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.851+0000 7f228f8b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f228810eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.851+0000 7f228f8b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f228810f020 con 0x7f228807cc90 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f228d64e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f228810eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f228d64e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f228810eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51840/0 (socket says 172.21.15.112:51840) 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f228d64e700 1 -- 172.21.15.112:0/2769634505 learned_addr learned my addr 172.21.15.112:0/2769634505 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f228d64e700 1 -- 172.21.15.112:0/2769634505 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228810f1a0 con 0x7f228807cc90 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f228d64e700 1 --2- 172.21.15.112:0/2769634505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f228810eae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f22840098a0 tx=0x7f2284009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c19da625a745e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f227ffff700 1 -- 172.21.15.112:0/2769634505 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284004160 con 0x7f228807cc90 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f227ffff700 1 -- 172.21.15.112:0/2769634505 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2284004300 con 0x7f228807cc90 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.852+0000 7f227ffff700 1 -- 172.21.15.112:0/2769634505 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284011af0 con 0x7f228807cc90 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 -- 172.21.15.112:0/2769634505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 msgr2=0x7f228810eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:41.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 --2- 172.21.15.112:0/2769634505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f228810eae0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f22840098a0 tx=0x7f2284009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 -- 172.21.15.112:0/2769634505 shutdown_connections 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 --2- 172.21.15.112:0/2769634505 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f228810eae0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 -- 172.21.15.112:0/2769634505 >> 172.21.15.112:0/2769634505 conn(0x7f2288104130 msgr2=0x7f2288106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 -- 172.21.15.112:0/2769634505 shutdown_connections 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.853+0000 7f228f8b2700 1 -- 172.21.15.112:0/2769634505 wait complete. 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228f8b2700 1 Processor -- start 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228f8b2700 1 -- start start 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228f8b2700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f22881a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228f8b2700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f22881100b0 con 0x7f228807cc90 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228d64e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f22881a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228d64e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f22881a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51848/0 (socket says 172.21.15.112:51848) 2024-04-25T16:28:41.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228d64e700 1 -- 172.21.15.112:0/1728158071 learned_addr learned my addr 172.21.15.112:0/1728158071 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228d64e700 1 -- 172.21.15.112:0/1728158071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22881a0590 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.854+0000 7f228d64e700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f22881a0050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2284009fb0 tx=0x7f2284011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.855+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22840036a0 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.855+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f228401a430 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.855+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22881a0820 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.855+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2284019550 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.855+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a3430 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.856+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 49505+0+0 (secure 0 0 0) 0x7f2284019850 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.856+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2288064cf0 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.856+0000 7f227e7fc700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f227403d590 0x7f227403faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.856+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22840528b0 con 0x7f228807cc90 2024-04-25T16:28:41.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.856+0000 7f228ce4d700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f227403d590 0x7f227403faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.856+0000 7f228ce4d700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f227403d590 0x7f227403faa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2278006fd0 tx=0x7f2278006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:40.861+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2284023390 con 0x7f228807cc90 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.115+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f22881a3650 con 0x7f228807cc90 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.166+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f22840522d0 con 0x7f228807cc90 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.168+0000 7f227e7fc700 1 -- 172.21.15.112:0/1728158071 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7f228400fd80 con 0x7f228807cc90 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f227403d590 msgr2=0x7f227403faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f227403d590 0x7f227403faa0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2278006fd0 tx=0x7f2278006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 msgr2=0x7f22881a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f22881a0050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2284009fb0 tx=0x7f2284011940 comp rx=0 tx=0).stop 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 shutdown_connections 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7f227403d590 0x7f227403faa0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 --2- 172.21.15.112:0/1728158071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f228807cc90 0x7f22881a0050 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:41.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 >> 172.21.15.112:0/1728158071 conn(0x7f2288104130 msgr2=0x7f22881059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:41.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 shutdown_connections 2024-04-25T16:28:41.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:41.169+0000 7f228f8b2700 1 -- 172.21.15.112:0/1728158071 wait complete. 2024-04-25T16:28:42.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:42 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1728158071' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T16:28:42.226 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:42 smithi112 ceph-mon[29122]: mgrmap e5: x(active, since 15s) 2024-04-25T16:28:42.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:41.976+0000 7f33863fa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:28:42.227 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:42.066+0000 7f33863fa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:28:43.034 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:28:43.034 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T16:28:43.034 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T16:28:43.034 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.225+0000 7fdb150a0700 1 Processor -- start 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.225+0000 7fdb150a0700 1 -- start start 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.225+0000 7fdb150a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1010c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.225+0000 7fdb150a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010cc80 con 0x7fdb1007d700 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1010c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1010c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51868/0 (socket says 172.21.15.112:51868) 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0ed9d700 1 -- 172.21.15.112:0/1042096447 learned_addr learned my addr 172.21.15.112:0/1042096447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0ed9d700 1 -- 172.21.15.112:0/1042096447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1010ce00 con 0x7fdb1007d700 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0ed9d700 1 --2- 172.21.15.112:0/1042096447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1010c670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb040098a0 tx=0x7fdb04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f191020307f5a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0dd9b700 1 -- 172.21.15.112:0/1042096447 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb04004160 con 0x7fdb1007d700 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0dd9b700 1 -- 172.21.15.112:0/1042096447 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdb04004300 con 0x7fdb1007d700 2024-04-25T16:28:43.035 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.226+0000 7fdb0dd9b700 1 -- 172.21.15.112:0/1042096447 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb04011af0 con 0x7fdb1007d700 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 -- 172.21.15.112:0/1042096447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 msgr2=0x7fdb1010c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 --2- 172.21.15.112:0/1042096447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1010c670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdb040098a0 tx=0x7fdb04009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 -- 172.21.15.112:0/1042096447 shutdown_connections 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 --2- 172.21.15.112:0/1042096447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1010c670 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 -- 172.21.15.112:0/1042096447 >> 172.21.15.112:0/1042096447 conn(0x7fdb101040c0 msgr2=0x7fdb10106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 -- 172.21.15.112:0/1042096447 shutdown_connections 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.227+0000 7fdb150a0700 1 -- 172.21.15.112:0/1042096447 wait complete. 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.228+0000 7fdb150a0700 1 Processor -- start 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.228+0000 7fdb150a0700 1 -- start start 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb150a0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1019de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb150a0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1010de80 con 0x7fdb1007d700 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb0ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1019de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb0ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1019de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51876/0 (socket says 172.21.15.112:51876) 2024-04-25T16:28:43.036 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb0ed9d700 1 -- 172.21.15.112:0/2985693511 learned_addr learned my addr 172.21.15.112:0/2985693511 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb0ed9d700 1 -- 172.21.15.112:0/2985693511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1019e360 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdb0ed9d700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1019de20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb040117b0 tx=0x7fdb04011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.229+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb04003af0 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.230+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdb04020070 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.230+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1019e5f0 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.230+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb04019630 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.230+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb101a1200 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.230+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7fdb040198a0 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.231+0000 7fdaf7fff700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 0x7fdaf803fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.231+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdb04059060 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.231+0000 7fdb0e59c700 1 -- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 msgr2=0x7fdaf803fbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.231+0000 7fdb0e59c700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 0x7fdaf803fbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.231+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb10064cf0 con 0x7fdb1007d700 2024-04-25T16:28:43.037 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.237+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb04018350 con 0x7fdb1007d700 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.432+0000 7fdb0e59c700 1 -- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 msgr2=0x7fdaf803fbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.432+0000 7fdb0e59c700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 0x7fdaf803fbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.535+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdb101a1420 con 0x7fdb1007d700 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.535+0000 7fdaf7fff700 1 -- 172.21.15.112:0/2985693511 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fdb040042c0 con 0x7fdb1007d700 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 msgr2=0x7fdaf803fbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 0x7fdaf803fbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 msgr2=0x7fdb1019de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1019de20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdb040117b0 tx=0x7fdb04011b70 comp rx=0 tx=0).stop 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 shutdown_connections 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7fdaf803d6d0 0x7fdaf803fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 --2- 172.21.15.112:0/2985693511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb1007d700 0x7fdb1019de20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 >> 172.21.15.112:0/2985693511 conn(0x7fdb101040c0 msgr2=0x7fdb10104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:43.038 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 shutdown_connections 2024-04-25T16:28:43.039 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:42.538+0000 7fdb150a0700 1 -- 172.21.15.112:0/2985693511 wait complete. 2024-04-25T16:28:43.039 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-04-25T16:28:43.039 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 5... 2024-04-25T16:28:43.411 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:43 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2985693511' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T16:28:43.411 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:43.308+0000 7f33863fa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:28:43.662 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:43.491+0000 7f33863fa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:28:43.662 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:43.662+0000 7f33863fa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:28:44.250 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:43.954+0000 7f33863fa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:28:44.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:44.048+0000 7f33863fa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:28:44.251 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:44.250+0000 7f33863fa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:28:44.597 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:44.447+0000 7f33863fa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:28:45.097 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:45.046+0000 7f33863fa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:28:45.769 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:45.471+0000 7f33863fa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:28:45.770 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:45.568+0000 7f33863fa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:28:45.770 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:45.769+0000 7f33863fa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:28:46.597 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:46.288+0000 7f33863fa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:28:46.597 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:46.380+0000 7f33863fa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:28:46.972 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:46.627+0000 7f33863fa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:28:46.972 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:46.760+0000 7f33863fa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:28:46.972 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:46.855+0000 7f33863fa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:28:46.972 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:46.972+0000 7f33863fa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:28:47.596 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:47.272+0000 7f33863fa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:28:48.143 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:47.871+0000 7f33863fa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:28:48.143 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:28:48.089+0000 7f33863fa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: Active manager daemon x restarted 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: Activating manager daemon x 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: mgrmap e6: x(active, starting, since 0.00379947s) 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T16:28:48.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T16:28:48.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T16:28:48.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: Manager daemon x is now available 2024-04-25T16:28:48.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:48.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:48.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:28:48.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:48 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: Found migration_current of "None". Setting to last migration. 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:28:49.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:49 smithi112 ceph-mon[29122]: mgrmap e7: x(active, since 1.00719s) 2024-04-25T16:28:49.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:28:49.816 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.438+0000 7ff04f10c700 1 Processor -- start 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.438+0000 7ff04f10c700 1 -- start start 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.438+0000 7ff04f10c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04810c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.438+0000 7ff04f10c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff04810ccf0 con 0x7ff04807d680 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.438+0000 7ff04cea8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04810c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff04cea8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04810c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51884/0 (socket says 172.21.15.112:51884) 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff04cea8700 1 -- 172.21.15.112:0/3947542993 learned_addr learned my addr 172.21.15.112:0/3947542993 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff04cea8700 1 -- 172.21.15.112:0/3947542993 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04810ce70 con 0x7ff04807d680 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff04cea8700 1 --2- 172.21.15.112:0/3947542993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04810c720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0098a0 tx=0x7ff03c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f759d9a07a59d9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff0477fe700 1 -- 172.21.15.112:0/3947542993 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03c004160 con 0x7ff04807d680 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff0477fe700 1 -- 172.21.15.112:0/3947542993 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff03c004300 con 0x7ff04807d680 2024-04-25T16:28:49.817 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.439+0000 7ff0477fe700 1 -- 172.21.15.112:0/3947542993 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03c011af0 con 0x7ff04807d680 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 -- 172.21.15.112:0/3947542993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 msgr2=0x7ff04810c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 --2- 172.21.15.112:0/3947542993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04810c720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0098a0 tx=0x7ff03c009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 -- 172.21.15.112:0/3947542993 shutdown_connections 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 --2- 172.21.15.112:0/3947542993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04810c720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 -- 172.21.15.112:0/3947542993 >> 172.21.15.112:0/3947542993 conn(0x7ff0481040d0 msgr2=0x7ff048106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 -- 172.21.15.112:0/3947542993 shutdown_connections 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.440+0000 7ff04f10c700 1 -- 172.21.15.112:0/3947542993 wait complete. 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04f10c700 1 Processor -- start 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04f10c700 1 -- start start 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04f10c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04f10c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff04810def0 con 0x7ff04807d680 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04cea8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:49.818 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04cea8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51890/0 (socket says 172.21.15.112:51890) 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04cea8700 1 -- 172.21.15.112:0/1798935949 learned_addr learned my addr 172.21.15.112:0/1798935949 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04cea8700 1 -- 172.21.15.112:0/1798935949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04819e390 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.441+0000 7ff04cea8700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04819de50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009fb0 tx=0x7ff03c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.442+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03c0036a0 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.442+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff03c01a430 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.442+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04819e620 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.442+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff03c019550 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.442+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0481a1230 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.443+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 49516+0+0 (secure 0 0 0) 0x7ff03c019850 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.443+0000 7ff045ffb700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.443+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 --> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff038040200 con 0x7ff03803d5e0 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.443+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff03c0536a0 con 0x7ff04807d680 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.443+0000 7ff047fff700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 msgr2=0x7ff03803faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:49.819 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.443+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.644+0000 7ff047fff700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 msgr2=0x7ff03803faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:43.644+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:44.045+0000 7ff047fff700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 msgr2=0x7ff03803faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:44.045+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:44.846+0000 7ff047fff700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 msgr2=0x7ff03803faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:44.846+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:46.447+0000 7ff047fff700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 msgr2=0x7ff03803faf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/1130097819 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:46.447+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:48.093+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 6) ==== 49283+0+0 (secure 0 0 0) 0x7ff03c00fd80 con 0x7ff04807d680 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:48.093+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 msgr2=0x7ff03803faf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:48.093+0000 7ff045ffb700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.095+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 7) ==== 49410+0+0 (secure 0 0 0) 0x7ff03c018350 con 0x7ff04807d680 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.095+0000 7ff045ffb700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7ff038040f20 0x7ff038043340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.095+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff03c02be50 con 0x7ff038040f20 2024-04-25T16:28:49.820 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.096+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7ff038040f20 0x7ff038043340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.097+0000 7ff047fff700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7ff038040f20 0x7ff038043340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff034003a10 tx=0x7ff034009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.097+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ff03c02be50 con 0x7ff038040f20 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.106+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff048064cf0 con 0x7ff038040f20 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.106+0000 7ff045ffb700 1 -- 172.21.15.112:0/1798935949 <== mgr.14120 v2:172.21.15.112:6800/108124461 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff048064cf0 con 0x7ff038040f20 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7ff038040f20 msgr2=0x7ff038043340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7ff038040f20 0x7ff038043340 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff034003a10 tx=0x7ff034009290 comp rx=0 tx=0).stop 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 msgr2=0x7ff04819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04819de50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff03c009fb0 tx=0x7ff03c011940 comp rx=0 tx=0).stop 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 shutdown_connections 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7ff038040f20 0x7ff038043340 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff04807d680 0x7ff04819de50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 --2- 172.21.15.112:0/1798935949 >> [v2:172.21.15.112:6800/1130097819,v1:172.21.15.112:6801/1130097819] conn(0x7ff03803d5e0 0x7ff03803faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 >> 172.21.15.112:0/1798935949 conn(0x7ff0481040d0 msgr2=0x7ff048104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 shutdown_connections 2024-04-25T16:28:49.821 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:49.107+0000 7ff04f10c700 1 -- 172.21.15.112:0/1798935949 wait complete. 2024-04-25T16:28:49.822 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 5 is available 2024-04-25T16:28:49.822 INFO:teuthology.orchestra.run.smithi112.stdout:Setting orchestrator backend to cephadm... 2024-04-25T16:28:50.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:50 smithi112 ceph-mon[29122]: [25/Apr/2024:16:28:48] ENGINE Bus STARTING 2024-04-25T16:28:50.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:50 smithi112 ceph-mon[29122]: [25/Apr/2024:16:28:48] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T16:28:50.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:50 smithi112 ceph-mon[29122]: [25/Apr/2024:16:28:48] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T16:28:50.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:50 smithi112 ceph-mon[29122]: [25/Apr/2024:16:28:48] ENGINE Bus STARTED 2024-04-25T16:28:50.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:50 smithi112 ceph-mon[29122]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T16:28:50.209 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:50 smithi112 ceph-mon[29122]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T16:28:51.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.256+0000 7f247b4f5700 1 Processor -- start 2024-04-25T16:28:51.128 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.256+0000 7f247b4f5700 1 -- start start 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f247b4f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247410eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f247b4f5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f247410f020 con 0x7f247407cc90 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f2479291700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247410eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f2479291700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247410eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49786/0 (socket says 172.21.15.112:49786) 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f2479291700 1 -- 172.21.15.112:0/3802885582 learned_addr learned my addr 172.21.15.112:0/3802885582 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f2479291700 1 -- 172.21.15.112:0/3802885582 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247410f1a0 con 0x7f247407cc90 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f2479291700 1 --2- 172.21.15.112:0/3802885582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247410eae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2470009e80 tx=0x7f247000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef1ee4b7ee14ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f246bfff700 1 -- 172.21.15.112:0/3802885582 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2470017070 con 0x7f247407cc90 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f246bfff700 1 -- 172.21.15.112:0/3802885582 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f247000b8c0 con 0x7f247407cc90 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.257+0000 7f246bfff700 1 -- 172.21.15.112:0/3802885582 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2470004490 con 0x7f247407cc90 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 -- 172.21.15.112:0/3802885582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 msgr2=0x7f247410eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 --2- 172.21.15.112:0/3802885582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247410eae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2470009e80 tx=0x7f247000b330 comp rx=0 tx=0).stop 2024-04-25T16:28:51.129 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 -- 172.21.15.112:0/3802885582 shutdown_connections 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 --2- 172.21.15.112:0/3802885582 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247410eae0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 -- 172.21.15.112:0/3802885582 >> 172.21.15.112:0/3802885582 conn(0x7f2474104130 msgr2=0x7f2474106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 -- 172.21.15.112:0/3802885582 shutdown_connections 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.258+0000 7f247b4f5700 1 -- 172.21.15.112:0/3802885582 wait complete. 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f247b4f5700 1 Processor -- start 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f247b4f5700 1 -- start start 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f247b4f5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247419fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f247b4f5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f24741100b0 con 0x7f247407cc90 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f2479291700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247419fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f2479291700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247419fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49788/0 (socket says 172.21.15.112:49788) 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.259+0000 7f2479291700 1 -- 172.21.15.112:0/254278407 learned_addr learned my addr 172.21.15.112:0/254278407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f2479291700 1 -- 172.21.15.112:0/254278407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24741a0530 con 0x7f247407cc90 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f2479291700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247419fff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2470004220 tx=0x7f247000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.130 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2470020020 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2470012430 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24741a07c0 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2470017040 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.260+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741a33d0 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.261+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f2470023020 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.261+0000 7f246a7fc700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f246003d630 0x7f246003fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.261+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f247000f050 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.261+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2474064cf0 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.261+0000 7f2478a90700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f246003d630 0x7f246003fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.262+0000 7f2478a90700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f246003d630 0x7f246003fb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2464006fd0 tx=0x7f2464006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.267+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f247002a020 con 0x7f247407cc90 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.481+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f24741060f0 con 0x7f246003d630 2024-04-25T16:28:51.131 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.487+0000 7f246a7fc700 1 -- 172.21.15.112:0/254278407 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24741060f0 con 0x7f246003d630 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f246003d630 msgr2=0x7f246003fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f246003d630 0x7f246003fb40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2464006fd0 tx=0x7f2464006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 msgr2=0x7f247419fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247419fff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2470004220 tx=0x7f247000bd50 comp rx=0 tx=0).stop 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 shutdown_connections 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f246003d630 0x7f246003fb40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 --2- 172.21.15.112:0/254278407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f247407cc90 0x7f247419fff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.489+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 >> 172.21.15.112:0/254278407 conn(0x7f2474104130 msgr2=0x7f24741059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.490+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 shutdown_connections 2024-04-25T16:28:51.132 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:50.490+0000 7f247b4f5700 1 -- 172.21.15.112:0/254278407 wait complete. 2024-04-25T16:28:51.372 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:51 smithi112 ceph-mon[29122]: mgrmap e8: x(active, since 2s) 2024-04-25T16:28:51.372 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:51 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:51.372 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:51 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:28:52.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:52 smithi112 ceph-mon[29122]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:28:52.359 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.577+0000 7fe763ac6700 1 Processor -- start 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.577+0000 7fe763ac6700 1 -- start start 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe763ac6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe763ac6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c109420 con 0x7fe75c108a20 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe761862700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe761862700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49798/0 (socket says 172.21.15.112:49798) 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe761862700 1 -- 172.21.15.112:0/309915308 learned_addr learned my addr 172.21.15.112:0/309915308 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe761862700 1 -- 172.21.15.112:0/309915308 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c109b50 con 0x7fe75c108a20 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe761862700 1 --2- 172.21.15.112:0/309915308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c108e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0098a0 tx=0x7fe74c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564872010348d664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe760860700 1 -- 172.21.15.112:0/309915308 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c004160 con 0x7fe75c108a20 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe760860700 1 -- 172.21.15.112:0/309915308 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe74c004300 con 0x7fe75c108a20 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.578+0000 7fe760860700 1 -- 172.21.15.112:0/309915308 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c011af0 con 0x7fe75c108a20 2024-04-25T16:28:52.360 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 -- 172.21.15.112:0/309915308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 msgr2=0x7fe75c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 --2- 172.21.15.112:0/309915308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c108e30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0098a0 tx=0x7fe74c009bb0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 -- 172.21.15.112:0/309915308 shutdown_connections 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 --2- 172.21.15.112:0/309915308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c108e30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 -- 172.21.15.112:0/309915308 >> 172.21.15.112:0/309915308 conn(0x7fe75c1040e0 msgr2=0x7fe75c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 -- 172.21.15.112:0/309915308 shutdown_connections 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.579+0000 7fe763ac6700 1 -- 172.21.15.112:0/309915308 wait complete. 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe763ac6700 1 Processor -- start 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe763ac6700 1 -- start start 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe763ac6700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c07c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe763ac6700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c10cf00 con 0x7fe75c108a20 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe761862700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c07c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe761862700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c07c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49808/0 (socket says 172.21.15.112:49808) 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.580+0000 7fe761862700 1 -- 172.21.15.112:0/2907832583 learned_addr learned my addr 172.21.15.112:0/2907832583 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe761862700 1 -- 172.21.15.112:0/2907832583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75c07ceb0 con 0x7fe75c108a20 2024-04-25T16:28:52.361 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe761862700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c07c970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009fb0 tx=0x7fe74c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c0036a0 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe74c003cd0 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe75c1a0c20 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe74c01a430 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.581+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe75c1a10a0 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.582+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fe74c01a6a0 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.582+0000 7fe752ffd700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe74803d690 0x7fe74803fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.582+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe74c053900 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.582+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe75c064cf0 con 0x7fe75c108a20 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.582+0000 7fe761061700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe74803d690 0x7fe74803fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.583+0000 7fe761061700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe74803d690 0x7fe74803fba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe758006fd0 tx=0x7fe758006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:52.362 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.588+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe74c017e90 con 0x7fe75c108a20 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.788+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe75c106040 con 0x7fe74803d690 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.789+0000 7fe752ffd700 1 -- 172.21.15.112:0/2907832583 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe75c106040 con 0x7fe74803d690 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.791+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe74803d690 msgr2=0x7fe74803fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.791+0000 7fe763ac6700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe74803d690 0x7fe74803fba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe758006fd0 tx=0x7fe758006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.791+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 msgr2=0x7fe75c07c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.791+0000 7fe763ac6700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c07c970 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009fb0 tx=0x7fe74c011890 comp rx=0 tx=0).stop 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.792+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 shutdown_connections 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.792+0000 7fe763ac6700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe74803d690 0x7fe74803fba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.792+0000 7fe763ac6700 1 --2- 172.21.15.112:0/2907832583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe75c108a20 0x7fe75c07c970 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.792+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 >> 172.21.15.112:0/2907832583 conn(0x7fe75c1040e0 msgr2=0x7fe75c105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.792+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 shutdown_connections 2024-04-25T16:28:52.363 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:51.792+0000 7fe763ac6700 1 -- 172.21.15.112:0/2907832583 wait complete. 2024-04-25T16:28:52.364 INFO:teuthology.orchestra.run.smithi112.stdout:Generating ssh key... 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-mon[29122]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: Generating public/private rsa key pair. 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: Your identification has been saved in /tmp/tmpi_ijk_y7/key. 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: Your public key has been saved in /tmp/tmpi_ijk_y7/key.pub. 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: The key fingerprint is: 2024-04-25T16:28:53.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: SHA256:CFWA5q1gTtCzTSQALmsL4mykqTJIr23CNSiXWf15B64 ceph-a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: The key's randomart image is: 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: +---[RSA 3072]----+ 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |+o....oo. | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |o o.+. | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |.o B.o | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |..= +.o. . | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |+*.= ...So . | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |X+*o. o o . | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |B*o . o . | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |*o.o E | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: |oo+. | 2024-04-25T16:28:53.348 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:28:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: +----[SHA256]-----+ 2024-04-25T16:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.779+0000 7f410f712700 1 Processor -- start 2024-04-25T16:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.779+0000 7f410f712700 1 -- start start 2024-04-25T16:28:53.639 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410f712700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410810eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410f712700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f410810f020 con 0x7f410807cc90 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410d4ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410810eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410d4ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410810eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49816/0 (socket says 172.21.15.112:49816) 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410d4ae700 1 -- 172.21.15.112:0/2845681175 learned_addr learned my addr 172.21.15.112:0/2845681175 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410d4ae700 1 -- 172.21.15.112:0/2845681175 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410810f1a0 con 0x7f410807cc90 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f410d4ae700 1 --2- 172.21.15.112:0/2845681175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410810eae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4100009e80 tx=0x7f410000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a41e111ad385653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f4107fff700 1 -- 172.21.15.112:0/2845681175 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100017070 con 0x7f410807cc90 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f4107fff700 1 -- 172.21.15.112:0/2845681175 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f410000b8c0 con 0x7f410807cc90 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.780+0000 7f4107fff700 1 -- 172.21.15.112:0/2845681175 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100004490 con 0x7f410807cc90 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 -- 172.21.15.112:0/2845681175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 msgr2=0x7f410810eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 --2- 172.21.15.112:0/2845681175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410810eae0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4100009e80 tx=0x7f410000b330 comp rx=0 tx=0).stop 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 -- 172.21.15.112:0/2845681175 shutdown_connections 2024-04-25T16:28:53.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 --2- 172.21.15.112:0/2845681175 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410810eae0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 -- 172.21.15.112:0/2845681175 >> 172.21.15.112:0/2845681175 conn(0x7f4108104130 msgr2=0x7f4108106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 -- 172.21.15.112:0/2845681175 shutdown_connections 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.781+0000 7f410f712700 1 -- 172.21.15.112:0/2845681175 wait complete. 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410f712700 1 Processor -- start 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410f712700 1 -- start start 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410f712700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410819fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410f712700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41081100b0 con 0x7f410807cc90 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410d4ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410819fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410d4ae700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410819fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49830/0 (socket says 172.21.15.112:49830) 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410d4ae700 1 -- 172.21.15.112:0/1789712377 learned_addr learned my addr 172.21.15.112:0/1789712377 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.782+0000 7f410d4ae700 1 -- 172.21.15.112:0/1789712377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41081a0530 con 0x7f410807cc90 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.783+0000 7f410d4ae700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410819fff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4100004220 tx=0x7f410000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.783+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100020020 con 0x7f410807cc90 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.783+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4100012430 con 0x7f410807cc90 2024-04-25T16:28:53.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.783+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41081a07c0 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.783+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4100017040 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.783+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41081a33d0 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.784+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f4100023020 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.784+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4108064cf0 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.784+0000 7f41067fc700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f40f003d680 0x7f40f003fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.784+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f410000f050 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.784+0000 7f410ccad700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f40f003d680 0x7f40f003fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.784+0000 7f410ccad700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f40f003d680 0x7f40f003fb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.789+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f410002a020 con 0x7f410807cc90 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:52.989+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f41081060f0 con 0x7f40f003d680 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.145+0000 7f41067fc700 1 -- 172.21.15.112:0/1789712377 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f41081060f0 con 0x7f40f003d680 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.147+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f40f003d680 msgr2=0x7f40f003fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.147+0000 7f410f712700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f40f003d680 0x7f40f003fb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f40fc006fd0 tx=0x7f40fc006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.147+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 msgr2=0x7f410819fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.147+0000 7f410f712700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410819fff0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4100004220 tx=0x7f410000bdc0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.148+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 shutdown_connections 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.148+0000 7f410f712700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f40f003d680 0x7f40f003fb90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.148+0000 7f410f712700 1 --2- 172.21.15.112:0/1789712377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f410807cc90 0x7f410819fff0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.148+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 >> 172.21.15.112:0/1789712377 conn(0x7f4108104130 msgr2=0x7f41081059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.148+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 shutdown_connections 2024-04-25T16:28:53.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:53.148+0000 7f410f712700 1 -- 172.21.15.112:0/1789712377 wait complete. 2024-04-25T16:28:54.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:54 smithi112 ceph-mon[29122]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:28:54.268 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:54 smithi112 ceph-mon[29122]: Generating ssh key... 2024-04-25T16:28:54.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:28:54.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.042+0000 7f7543221700 1 Processor -- start 2024-04-25T16:28:54.896 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.042+0000 7f7543221700 1 -- start start 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.042+0000 7f7543221700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c109f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.042+0000 7f7543221700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10a510 con 0x7f753c109b10 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f7540fbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c109f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f7540fbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c109f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49832/0 (socket says 172.21.15.112:49832) 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f7540fbd700 1 -- 172.21.15.112:0/2508507200 learned_addr learned my addr 172.21.15.112:0/2508507200 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f7540fbd700 1 -- 172.21.15.112:0/2508507200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c10ac40 con 0x7f753c109b10 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f7540fbd700 1 --2- 172.21.15.112:0/2508507200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c109f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7530009e80 tx=0x7f753000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881c6dfee4839fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f753b7fe700 1 -- 172.21.15.112:0/2508507200 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7530017070 con 0x7f753c109b10 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f753b7fe700 1 -- 172.21.15.112:0/2508507200 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f753000b8c0 con 0x7f753c109b10 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.043+0000 7f753b7fe700 1 -- 172.21.15.112:0/2508507200 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7530004490 con 0x7f753c109b10 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 -- 172.21.15.112:0/2508507200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 msgr2=0x7f753c109f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 --2- 172.21.15.112:0/2508507200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c109f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7530009e80 tx=0x7f753000b330 comp rx=0 tx=0).stop 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 -- 172.21.15.112:0/2508507200 shutdown_connections 2024-04-25T16:28:54.897 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 --2- 172.21.15.112:0/2508507200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c109f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 -- 172.21.15.112:0/2508507200 >> 172.21.15.112:0/2508507200 conn(0x7f753c1051d0 msgr2=0x7f753c107620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 -- 172.21.15.112:0/2508507200 shutdown_connections 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.044+0000 7f7543221700 1 -- 172.21.15.112:0/2508507200 wait complete. 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7543221700 1 Processor -- start 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7543221700 1 -- start start 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7543221700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7543221700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10dff0 con 0x7f753c109b10 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7540fbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7540fbd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c19de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49838/0 (socket says 172.21.15.112:49838) 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7540fbd700 1 -- 172.21.15.112:0/2136478880 learned_addr learned my addr 172.21.15.112:0/2136478880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7540fbd700 1 -- 172.21.15.112:0/2136478880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19e360 con 0x7f753c109b10 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7540fbd700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c19de20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7530009fd0 tx=0x7f753000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:54.898 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.045+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7530020020 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.046+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7530012430 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.046+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7530017040 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.046+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19e5f0 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.046+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1a1200 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.047+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f7530023020 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.047+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c064cf0 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.047+0000 7f7539ffb700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f752c03d5e0 0x7f752c03faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.047+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f753000f050 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.047+0000 7f753bfff700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f752c03d5e0 0x7f752c03faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.047+0000 7f753bfff700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f752c03d5e0 0x7f752c03faf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7528006fd0 tx=0x7f7528006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.052+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f753002a020 con 0x7f753c109b10 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.254+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f753c107100 con 0x7f752c03d5e0 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.255+0000 7f7539ffb700 1 -- 172.21.15.112:0/2136478880 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f753c107100 con 0x7f752c03d5e0 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f752c03d5e0 msgr2=0x7f752c03faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.899 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f752c03d5e0 0x7f752c03faf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7528006fd0 tx=0x7f7528006e40 comp rx=0 tx=0).stop 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 msgr2=0x7f753c19de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c19de20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7530009fd0 tx=0x7f753000bd50 comp rx=0 tx=0).stop 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 shutdown_connections 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f752c03d5e0 0x7f752c03faf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 --2- 172.21.15.112:0/2136478880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f753c109b10 0x7f753c19de20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 >> 172.21.15.112:0/2136478880 conn(0x7f753c1051d0 msgr2=0x7f753c1069f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 shutdown_connections 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:54.257+0000 7f7543221700 1 -- 172.21.15.112:0/2136478880 wait complete. 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T16:28:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:Adding host smithi112... 2024-04-25T16:28:55.551 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:55 smithi112 ceph-mon[29122]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:28:56.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:56 smithi112 ceph-mon[29122]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:28:56.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:28:56 smithi112 ceph-mon[29122]: Deploying cephadm binary to smithi112 2024-04-25T16:29:00.197 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Added host 'smithi112' with addr '172.21.15.112' 2024-04-25T16:29:00.197 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.288+0000 7fe7be465700 1 Processor -- start 2024-04-25T16:29:00.197 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.288+0000 7fe7be465700 1 -- start start 2024-04-25T16:29:00.197 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7be465700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b8108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:00.197 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7be465700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8109420 con 0x7fe7b8108a20 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b8108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b8108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49840/0 (socket says 172.21.15.112:49840) 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b7fff700 1 -- 172.21.15.112:0/1307560124 learned_addr learned my addr 172.21.15.112:0/1307560124 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b7fff700 1 -- 172.21.15.112:0/1307560124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b8109b50 con 0x7fe7b8108a20 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b7fff700 1 --2- 172.21.15.112:0/1307560124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b8108e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80098a0 tx=0x7fe7a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a0255d5f6c6af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b77fe700 1 -- 172.21.15.112:0/1307560124 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a8004160 con 0x7fe7b8108a20 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b77fe700 1 -- 172.21.15.112:0/1307560124 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7a8004300 con 0x7fe7b8108a20 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.289+0000 7fe7b77fe700 1 -- 172.21.15.112:0/1307560124 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a8011af0 con 0x7fe7b8108a20 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 -- 172.21.15.112:0/1307560124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 msgr2=0x7fe7b8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:00.198 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 --2- 172.21.15.112:0/1307560124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b8108e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7a80098a0 tx=0x7fe7a8009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 -- 172.21.15.112:0/1307560124 shutdown_connections 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 --2- 172.21.15.112:0/1307560124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b8108e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 -- 172.21.15.112:0/1307560124 >> 172.21.15.112:0/1307560124 conn(0x7fe7b81040e0 msgr2=0x7fe7b8106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 -- 172.21.15.112:0/1307560124 shutdown_connections 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.290+0000 7fe7be465700 1 -- 172.21.15.112:0/1307560124 wait complete. 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7be465700 1 Processor -- start 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7be465700 1 -- start start 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7be465700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b807c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7be465700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b810cf00 con 0x7fe7b8108a20 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b807c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:00.199 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7b7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b807c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49844/0 (socket says 172.21.15.112:49844) 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.291+0000 7fe7b7fff700 1 -- 172.21.15.112:0/2685998825 learned_addr learned my addr 172.21.15.112:0/2685998825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7b7fff700 1 -- 172.21.15.112:0/2685998825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b807ce90 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7b7fff700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b807c950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009fb0 tx=0x7fe7a8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a80036a0 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe7a8003cd0 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7a801a430 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81a0c10 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.292+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a1090 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.293+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fe7a801a6a0 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.293+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b8064cf0 con 0x7fe7b8108a20 2024-04-25T16:29:00.200 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.293+0000 7fe7b5ffb700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe7a403d630 0x7fe7a403fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.293+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7a80526f0 con 0x7fe7b8108a20 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.293+0000 7fe7affff700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe7a403d630 0x7fe7a403fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.294+0000 7fe7affff700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe7a403d630 0x7fe7a403fb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006fd0 tx=0x7fe7a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.298+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7a8012040 con 0x7fe7b8108a20 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:55.503+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi112", "addr": "172.21.15.112", "target": ["mon-mgr", ""]}) -- 0x7fe7b8105fe0 con 0x7fe7a403d630 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.604+0000 7fe7b5ffb700 1 -- 172.21.15.112:0/2685998825 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe7b8105fe0 con 0x7fe7a403d630 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe7a403d630 msgr2=0x7fe7a403fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe7a403d630 0x7fe7a403fb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006fd0 tx=0x7fe7a0006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:00.201 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 msgr2=0x7fe7b807c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b807c950 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009fb0 tx=0x7fe7a8011890 comp rx=0 tx=0).stop 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 shutdown_connections 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe7a403d630 0x7fe7a403fb40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 --2- 172.21.15.112:0/2685998825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe7b8108a20 0x7fe7b807c950 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 >> 172.21.15.112:0/2685998825 conn(0x7fe7b81040e0 msgr2=0x7fe7b81058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 shutdown_connections 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:28:59.607+0000 7fe7be465700 1 -- 172.21.15.112:0/2685998825 wait complete. 2024-04-25T16:29:00.202 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mon service... 2024-04-25T16:29:00.838 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:00 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:00.839 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:00 smithi112 ceph-mon[29122]: Added host smithi112 2024-04-25T16:29:00.839 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:00 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:01.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T16:29:01.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.613+0000 7f34ef0e8700 1 Processor -- start 2024-04-25T16:29:01.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.613+0000 7f34ef0e8700 1 -- start start 2024-04-25T16:29:01.452 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ef0e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e8108e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ef0e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34e8109470 con 0x7f34e8108a70 2024-04-25T16:29:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ece84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e8108e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ece84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e8108e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34608/0 (socket says 172.21.15.112:34608) 2024-04-25T16:29:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ece84700 1 -- 172.21.15.112:0/3659647960 learned_addr learned my addr 172.21.15.112:0/3659647960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ece84700 1 -- 172.21.15.112:0/3659647960 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e8109ba0 con 0x7f34e8108a70 2024-04-25T16:29:01.453 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.614+0000 7f34ece84700 1 --2- 172.21.15.112:0/3659647960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e8108e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0098a0 tx=0x7f34dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72edab29d74bb646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.615+0000 7f34e77fe700 1 -- 172.21.15.112:0/3659647960 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34dc004160 con 0x7f34e8108a70 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.615+0000 7f34e77fe700 1 -- 172.21.15.112:0/3659647960 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f34dc004300 con 0x7f34e8108a70 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.615+0000 7f34e77fe700 1 -- 172.21.15.112:0/3659647960 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34dc011af0 con 0x7f34e8108a70 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.615+0000 7f34ef0e8700 1 -- 172.21.15.112:0/3659647960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 msgr2=0x7f34e8108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.615+0000 7f34ef0e8700 1 --2- 172.21.15.112:0/3659647960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e8108e80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0098a0 tx=0x7f34dc009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 -- 172.21.15.112:0/3659647960 shutdown_connections 2024-04-25T16:29:01.454 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 --2- 172.21.15.112:0/3659647960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e8108e80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 -- 172.21.15.112:0/3659647960 >> 172.21.15.112:0/3659647960 conn(0x7f34e8104130 msgr2=0x7f34e8106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 -- 172.21.15.112:0/3659647960 shutdown_connections 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 -- 172.21.15.112:0/3659647960 wait complete. 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 Processor -- start 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 -- start start 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:01.455 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.616+0000 7f34ef0e8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f34e810cf50 con 0x7f34e8108a70 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ece84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ece84700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34622/0 (socket says 172.21.15.112:34622) 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ece84700 1 -- 172.21.15.112:0/768201209 learned_addr learned my addr 172.21.15.112:0/768201209 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ece84700 1 -- 172.21.15.112:0/768201209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e819d320 con 0x7f34e8108a70 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ece84700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e819cde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009fb0 tx=0x7f34dc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34dc0036a0 con 0x7f34e8108a70 2024-04-25T16:29:01.456 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f34dc003cd0 con 0x7f34e8108a70 2024-04-25T16:29:01.457 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e819d5b0 con 0x7f34e8108a70 2024-04-25T16:29:01.457 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f34dc01a430 con 0x7f34e8108a70 2024-04-25T16:29:01.457 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.617+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e81a01c0 con 0x7f34e8108a70 2024-04-25T16:29:01.457 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.618+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f34dc01a6a0 con 0x7f34e8108a70 2024-04-25T16:29:01.457 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.618+0000 7f34e5ffb700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f34d803d630 0x7f34d803fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:01.457 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.619+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f34dc053440 con 0x7f34e8108a70 2024-04-25T16:29:01.458 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.619+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34e8064cf0 con 0x7f34e8108a70 2024-04-25T16:29:01.458 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.619+0000 7f34e7fff700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f34d803d630 0x7f34d803fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:01.458 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.619+0000 7f34e7fff700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f34d803d630 0x7f34d803fb40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f34d4006fd0 tx=0x7f34d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:01.458 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.624+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34dc017e90 con 0x7f34e8108a70 2024-04-25T16:29:01.458 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.838+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f34e8106090 con 0x7f34d803d630 2024-04-25T16:29:01.458 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.843+0000 7f34e5ffb700 1 -- 172.21.15.112:0/768201209 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f34e8106090 con 0x7f34d803d630 2024-04-25T16:29:01.459 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f34d803d630 msgr2=0x7f34d803fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:01.459 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f34d803d630 0x7f34d803fb40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f34d4006fd0 tx=0x7f34d4006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:01.459 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 msgr2=0x7f34e819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:01.459 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e819cde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009fb0 tx=0x7f34dc011890 comp rx=0 tx=0).stop 2024-04-25T16:29:01.459 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 shutdown_connections 2024-04-25T16:29:01.459 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f34d803d630 0x7f34d803fb40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:01.460 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 --2- 172.21.15.112:0/768201209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f34e8108a70 0x7f34e819cde0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:01.460 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.845+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 >> 172.21.15.112:0/768201209 conn(0x7f34e8104130 msgr2=0x7f34e8105980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:01.460 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.846+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 shutdown_connections 2024-04-25T16:29:01.460 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:00.846+0000 7f34ef0e8700 1 -- 172.21.15.112:0/768201209 wait complete. 2024-04-25T16:29:01.460 INFO:teuthology.orchestra.run.smithi112.stdout:Deploying unmanaged mgr service... 2024-04-25T16:29:02.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:01 smithi112 ceph-mon[29122]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:02.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:01 smithi112 ceph-mon[29122]: Saving service mon spec with placement count:5 2024-04-25T16:29:02.091 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:01 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:03.049 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T16:29:03.055 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.864+0000 7fbce3e66700 1 Processor -- start 2024-04-25T16:29:03.055 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.864+0000 7fbce3e66700 1 -- start start 2024-04-25T16:29:03.055 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce3e66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc108250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:03.055 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce3e66700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc108860 con 0x7fbcdc107e40 2024-04-25T16:29:03.055 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce1c02700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc108250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce1c02700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc108250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34628/0 (socket says 172.21.15.112:34628) 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce1c02700 1 -- 172.21.15.112:0/2169415681 learned_addr learned my addr 172.21.15.112:0/2169415681 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce1c02700 1 -- 172.21.15.112:0/2169415681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc1089e0 con 0x7fbcdc107e40 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.865+0000 7fbce1c02700 1 --2- 172.21.15.112:0/2169415681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc108250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c327c8bbc556c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.866+0000 7fbce0c00700 1 -- 172.21.15.112:0/2169415681 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8004160 con 0x7fbcdc107e40 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.866+0000 7fbce0c00700 1 -- 172.21.15.112:0/2169415681 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbcd8004300 con 0x7fbcdc107e40 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.866+0000 7fbce0c00700 1 -- 172.21.15.112:0/2169415681 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8011af0 con 0x7fbcdc107e40 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.866+0000 7fbce3e66700 1 -- 172.21.15.112:0/2169415681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 msgr2=0x7fbcdc108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.866+0000 7fbce3e66700 1 --2- 172.21.15.112:0/2169415681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc108250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 -- 172.21.15.112:0/2169415681 shutdown_connections 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 --2- 172.21.15.112:0/2169415681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc108250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 -- 172.21.15.112:0/2169415681 >> 172.21.15.112:0/2169415681 conn(0x7fbcdc103840 msgr2=0x7fbcdc105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 -- 172.21.15.112:0/2169415681 shutdown_connections 2024-04-25T16:29:03.056 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 -- 172.21.15.112:0/2169415681 wait complete. 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 Processor -- start 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 -- start start 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.867+0000 7fbce3e66700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc10de70 con 0x7fbcdc107e40 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce1c02700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce1c02700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34636/0 (socket says 172.21.15.112:34636) 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce1c02700 1 -- 172.21.15.112:0/178950867 learned_addr learned my addr 172.21.15.112:0/178950867 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce1c02700 1 -- 172.21.15.112:0/178950867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc19e390 con 0x7fbcdc107e40 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce1c02700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc19de50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80117b0 tx=0x7fbcd8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8003af0 con 0x7fbcdc107e40 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc19e620 con 0x7fbcdc107e40 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbcd8020070 con 0x7fbcdc107e40 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8019630 con 0x7fbcdc107e40 2024-04-25T16:29:03.057 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.868+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc1a1230 con 0x7fbcdc107e40 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.869+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fbcd80198f0 con 0x7fbcdc107e40 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.869+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc064cf0 con 0x7fbcdc107e40 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.869+0000 7fbcd2ffd700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fbcc803d630 0x7fbcc803fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.869+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbcd8052970 con 0x7fbcdc107e40 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.870+0000 7fbce1401700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fbcc803d630 0x7fbcc803fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.870+0000 7fbce1401700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fbcc803d630 0x7fbcc803fb40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:01.875+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbcdc1089e0 con 0x7fbcdc107e40 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.091+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbcdc104810 con 0x7fbcc803d630 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.098+0000 7fbcd2ffd700 1 -- 172.21.15.112:0/178950867 <== mgr.14120 v2:172.21.15.112:6800/108124461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbcdc104810 con 0x7fbcc803d630 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fbcc803d630 msgr2=0x7fbcc803fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fbcc803d630 0x7fbcc803fb40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbccc006fd0 tx=0x7fbccc006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 msgr2=0x7fbcdc19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:03.058 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc19de50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80117b0 tx=0x7fbcd8011b70 comp rx=0 tx=0).stop 2024-04-25T16:29:03.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 shutdown_connections 2024-04-25T16:29:03.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fbcc803d630 0x7fbcc803fb40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:03.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 --2- 172.21.15.112:0/178950867 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcdc107e40 0x7fbcdc19de50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:03.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 >> 172.21.15.112:0/178950867 conn(0x7fbcdc103840 msgr2=0x7fbcdc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:03.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 shutdown_connections 2024-04-25T16:29:03.059 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:02.100+0000 7fbce3e66700 1 -- 172.21.15.112:0/178950867 wait complete. 2024-04-25T16:29:03.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:03 smithi112 ceph-mon[29122]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:03.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:03 smithi112 ceph-mon[29122]: Saving service mgr spec with placement count:2 2024-04-25T16:29:03.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:03 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45d2a03700 1 Processor -- start 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45d2a03700 1 -- start start 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45d2a03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc10ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45d2a03700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc07d670 con 0x7f45cc10a650 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc10ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc10ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34652/0 (socket says 172.21.15.112:34652) 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.491+0000 7f45cbfff700 1 -- 172.21.15.112:0/745240726 learned_addr learned my addr 172.21.15.112:0/745240726 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.492+0000 7f45cbfff700 1 -- 172.21.15.112:0/745240726 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc10cfb0 con 0x7f45cc10a650 2024-04-25T16:29:04.388 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.492+0000 7f45cbfff700 1 --2- 172.21.15.112:0/745240726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc10ca70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=995bcd734eade6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.492+0000 7f45caffd700 1 -- 172.21.15.112:0/745240726 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc004160 con 0x7f45cc10a650 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.492+0000 7f45caffd700 1 -- 172.21.15.112:0/745240726 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f45bc004300 con 0x7f45cc10a650 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.492+0000 7f45caffd700 1 -- 172.21.15.112:0/745240726 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc011af0 con 0x7f45cc10a650 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 -- 172.21.15.112:0/745240726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 msgr2=0x7f45cc10ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 --2- 172.21.15.112:0/745240726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc10ca70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098a0 tx=0x7f45bc009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 -- 172.21.15.112:0/745240726 shutdown_connections 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 --2- 172.21.15.112:0/745240726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc10ca70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 -- 172.21.15.112:0/745240726 >> 172.21.15.112:0/745240726 conn(0x7f45cc1040f0 msgr2=0x7f45cc106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 -- 172.21.15.112:0/745240726 shutdown_connections 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 -- 172.21.15.112:0/745240726 wait complete. 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.493+0000 7f45d2a03700 1 Processor -- start 2024-04-25T16:29:04.389 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45d2a03700 1 -- start start 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45d2a03700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45d2a03700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc10deb0 con 0x7f45cc10a650 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45cbfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc19ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34658/0 (socket says 172.21.15.112:34658) 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45cbfff700 1 -- 172.21.15.112:0/2943942035 learned_addr learned my addr 172.21.15.112:0/2943942035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45cbfff700 1 -- 172.21.15.112:0/2943942035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc19e330 con 0x7f45cc10a650 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.494+0000 7f45cbfff700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc19ddf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0117b0 tx=0x7f45bc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.495+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc003af0 con 0x7f45cc10a650 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.495+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f45bc020070 con 0x7f45cc10a650 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.495+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45bc019630 con 0x7f45cc10a650 2024-04-25T16:29:04.390 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.495+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc19e5c0 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.495+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1a11d0 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.496+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc064cf0 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.496+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7f45bc0198a0 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.496+0000 7f45c97fa700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f45b403d680 0x7f45b403fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.496+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f45bc0528a0 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.496+0000 7f45cb7fe700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f45b403d680 0x7f45b403fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.496+0000 7f45cb7fe700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f45b403d680 0x7f45b403fb90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f45c0006fd0 tx=0x7f45c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.501+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45bc016260 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.699+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f45cc1a1850 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.702+0000 7f45c97fa700 1 -- 172.21.15.112:0/2943942035 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f45bc018d30 con 0x7f45cc10a650 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f45b403d680 msgr2=0x7f45b403fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:04.391 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f45b403d680 0x7f45b403fb90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f45c0006fd0 tx=0x7f45c0006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 msgr2=0x7f45cc19ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc19ddf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0117b0 tx=0x7f45bc011b70 comp rx=0 tx=0).stop 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 shutdown_connections 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f45b403d680 0x7f45b403fb90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 --2- 172.21.15.112:0/2943942035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45cc10a650 0x7f45cc19ddf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.704+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 >> 172.21.15.112:0/2943942035 conn(0x7f45cc1040f0 msgr2=0x7f45cc104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.705+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 shutdown_connections 2024-04-25T16:29:04.392 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:03.705+0000 7f45d2a03700 1 -- 172.21.15.112:0/2943942035 wait complete. 2024-04-25T16:29:04.808 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:04 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2943942035' entity='client.admin' 2024-04-25T16:29:04.808 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:04 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:04.808 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:04 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:06.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:06 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/119647159' entity='client.admin' 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.815+0000 7facd02fa700 1 Processor -- start 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.815+0000 7facd02fa700 1 -- start start 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.815+0000 7facd02fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc810c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.815+0000 7facd02fa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facc810ccf0 con 0x7facc807d540 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7facce096700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc810c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7facce096700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc810c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34670/0 (socket says 172.21.15.112:34670) 2024-04-25T16:29:06.397 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7facce096700 1 -- 172.21.15.112:0/2544781944 learned_addr learned my addr 172.21.15.112:0/2544781944 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7facce096700 1 -- 172.21.15.112:0/2544781944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc810ce70 con 0x7facc807d540 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7facce096700 1 --2- 172.21.15.112:0/2544781944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc810c700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7328d3f52e71bf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7faccd094700 1 -- 172.21.15.112:0/2544781944 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facc4004160 con 0x7facc807d540 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7faccd094700 1 -- 172.21.15.112:0/2544781944 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc4004300 con 0x7facc807d540 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.816+0000 7faccd094700 1 -- 172.21.15.112:0/2544781944 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facc4011a20 con 0x7facc807d540 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 -- 172.21.15.112:0/2544781944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 msgr2=0x7facc810c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 --2- 172.21.15.112:0/2544781944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc810c700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7facc40098a0 tx=0x7facc4009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 -- 172.21.15.112:0/2544781944 shutdown_connections 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 --2- 172.21.15.112:0/2544781944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc810c700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 -- 172.21.15.112:0/2544781944 >> 172.21.15.112:0/2544781944 conn(0x7facc8104130 msgr2=0x7facc81065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 -- 172.21.15.112:0/2544781944 shutdown_connections 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.817+0000 7facd02fa700 1 -- 172.21.15.112:0/2544781944 wait complete. 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facd02fa700 1 Processor -- start 2024-04-25T16:29:06.398 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facd02fa700 1 -- start start 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facd02fa700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc8109e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facd02fa700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facc810df30 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facce096700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc8109e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facce096700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc8109e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34672/0 (socket says 172.21.15.112:34672) 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.818+0000 7facce096700 1 -- 172.21.15.112:0/119647159 learned_addr learned my addr 172.21.15.112:0/119647159 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facce096700 1 -- 172.21.15.112:0/119647159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc810a3b0 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facce096700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc8109e70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facc4000c00 tx=0x7facc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facc4003aa0 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facc401b8f0 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc8108600 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7facc4023890 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.819+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc8108ae0 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.820+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7facc402c3f0 con 0x7facc807d540 2024-04-25T16:29:06.399 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.820+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facc8064cf0 con 0x7facc807d540 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.820+0000 7facbf7fe700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7facb403d630 0x7facb403fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.820+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7facc4053cb0 con 0x7facc807d540 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.820+0000 7faccd895700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7facb403d630 0x7facb403fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.821+0000 7faccd895700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7facb403d630 0x7facb403fb40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facb8006fd0 tx=0x7facb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:04.826+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facc401f030 con 0x7facc807d540 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.040+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7facc8109240 con 0x7facc807d540 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.045+0000 7facbf7fe700 1 -- 172.21.15.112:0/119647159 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7facc40172b0 con 0x7facc807d540 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7facb403d630 msgr2=0x7facb403fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7facb403d630 0x7facb403fb40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7facb8006fd0 tx=0x7facb8006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 msgr2=0x7facc8109e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc8109e70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7facc4000c00 tx=0x7facc4011aa0 comp rx=0 tx=0).stop 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 shutdown_connections 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7facb403d630 0x7facb403fb40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 --2- 172.21.15.112:0/119647159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facc807d540 0x7facc8109e70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:06.400 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 >> 172.21.15.112:0/119647159 conn(0x7facc8104130 msgr2=0x7facc8104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:06.401 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 shutdown_connections 2024-04-25T16:29:06.401 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:05.048+0000 7facd02fa700 1 -- 172.21.15.112:0/119647159 wait complete. 2024-04-25T16:29:06.401 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling the dashboard module... 2024-04-25T16:29:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:07 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3811314691' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T16:29:08.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-mon[29122]: from='mgr.14120 172.21.15.112:0/3051762061' entity='mgr.x' 2024-04-25T16:29:08.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: ignoring --setuser ceph since I am not root 2024-04-25T16:29:08.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: ignoring --setgroup ceph since I am not root 2024-04-25T16:29:08.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:08.289+0000 7fb33f4f7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:29:08.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:08.476+0000 7fb33f4f7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:29:08.847 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:08.580+0000 7fb33f4f7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:29:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.825+0000 7fe6913ee700 1 Processor -- start 2024-04-25T16:29:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.825+0000 7fe6913ee700 1 -- start start 2024-04-25T16:29:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.825+0000 7fe6913ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.825+0000 7fe6913ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c109420 con 0x7fe68c108a20 2024-04-25T16:29:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe68affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:08.932 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe68affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56604/0 (socket says 172.21.15.112:56604) 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe68affd700 1 -- 172.21.15.112:0/1810061216 learned_addr learned my addr 172.21.15.112:0/1810061216 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe68affd700 1 -- 172.21.15.112:0/1810061216 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c109b50 con 0x7fe68c108a20 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe68affd700 1 --2- 172.21.15.112:0/1810061216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c108e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe684009e80 tx=0x7fe68400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80919d8eadb9c61d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe689ffb700 1 -- 172.21.15.112:0/1810061216 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684017070 con 0x7fe68c108a20 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe689ffb700 1 -- 172.21.15.112:0/1810061216 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68400b8c0 con 0x7fe68c108a20 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.826+0000 7fe689ffb700 1 -- 172.21.15.112:0/1810061216 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684004490 con 0x7fe68c108a20 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 -- 172.21.15.112:0/1810061216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 msgr2=0x7fe68c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 --2- 172.21.15.112:0/1810061216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c108e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe684009e80 tx=0x7fe68400b330 comp rx=0 tx=0).stop 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 -- 172.21.15.112:0/1810061216 shutdown_connections 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 --2- 172.21.15.112:0/1810061216 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c108e30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 -- 172.21.15.112:0/1810061216 >> 172.21.15.112:0/1810061216 conn(0x7fe68c1040e0 msgr2=0x7fe68c106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 -- 172.21.15.112:0/1810061216 shutdown_connections 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.827+0000 7fe6913ee700 1 -- 172.21.15.112:0/1810061216 wait complete. 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.828+0000 7fe6913ee700 1 Processor -- start 2024-04-25T16:29:08.933 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.828+0000 7fe6913ee700 1 -- start start 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe6913ee700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe6913ee700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c10cf00 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe68affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe68affd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56614/0 (socket says 172.21.15.112:56614) 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe68affd700 1 -- 172.21.15.112:0/3811314691 learned_addr learned my addr 172.21.15.112:0/3811314691 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe68affd700 1 -- 172.21.15.112:0/3811314691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c19d290 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe68affd700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c19cd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe684009fd0 tx=0x7fe68400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684020020 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe684003b50 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe684017040 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c19d520 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.829+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a0120 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.830+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 8) ==== 49516+0+0 (secure 0 0 0) 0x7fe684023020 con 0x7fe68c108a20 2024-04-25T16:29:08.934 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.831+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe68c064cf0 con 0x7fe68c108a20 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.831+0000 7fe673fff700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe67403d590 0x7fe67403faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.831+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe68400f050 con 0x7fe68c108a20 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.831+0000 7fe68a7fc700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe67403d590 0x7fe67403faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.831+0000 7fe68a7fc700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe67403d590 0x7fe67403faa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe680006fd0 tx=0x7fe680006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:06.836+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe68402a020 con 0x7fe68c108a20 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:07.098+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fe68c07c460 con 0x7fe68c108a20 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.087+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fe68402dd60 con 0x7fe68c108a20 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.089+0000 7fe673fff700 1 -- 172.21.15.112:0/3811314691 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fe684025400 con 0x7fe68c108a20 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe67403d590 msgr2=0x7fe67403faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe67403d590 0x7fe67403faa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe680006fd0 tx=0x7fe680006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 msgr2=0x7fe68c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c19cd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe684009fd0 tx=0x7fe68400bd30 comp rx=0 tx=0).stop 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 shutdown_connections 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fe67403d590 0x7fe67403faa0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:08.935 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 --2- 172.21.15.112:0/3811314691 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe68c108a20 0x7fe68c19cd50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:08.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 >> 172.21.15.112:0/3811314691 conn(0x7fe68c1040e0 msgr2=0x7fe68c105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:08.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 shutdown_connections 2024-04-25T16:29:08.936 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:08.090+0000 7fe6913ee700 1 -- 172.21.15.112:0/3811314691 wait complete. 2024-04-25T16:29:09.193 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:09 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3811314691' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T16:29:09.193 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:09 smithi112 ceph-mon[29122]: mgrmap e9: x(active, since 19s) 2024-04-25T16:29:09.193 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:08.889+0000 7fb33f4f7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:29:09.193 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:08.983+0000 7fb33f4f7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:29:10.366 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:10 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3254496035' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T16:29:10.366 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:10.192+0000 7fb33f4f7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:29:10.366 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:10.366+0000 7fb33f4f7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:29:10.658 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:10.537+0000 7fb33f4f7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.830+0000 7f10e2a6d700 1 Processor -- start 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.830+0000 7f10e2a6d700 1 -- start start 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.830+0000 7f10e2a6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc10a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.830+0000 7f10e2a6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc10a960 con 0x7f10dc109f60 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.831+0000 7f10e0809700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc10a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:10.667 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.831+0000 7f10e0809700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc10a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56624/0 (socket says 172.21.15.112:56624) 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.831+0000 7f10e0809700 1 -- 172.21.15.112:0/3966922558 learned_addr learned my addr 172.21.15.112:0/3966922558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.831+0000 7f10e0809700 1 -- 172.21.15.112:0/3966922558 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc108c80 con 0x7f10dc109f60 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.831+0000 7f10e0809700 1 --2- 172.21.15.112:0/3966922558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc10a370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098a0 tx=0x7f10d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a994b37e2033ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.832+0000 7f10db7fe700 1 -- 172.21.15.112:0/3966922558 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d0004160 con 0x7f10dc109f60 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.832+0000 7f10db7fe700 1 -- 172.21.15.112:0/3966922558 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d0004300 con 0x7f10dc109f60 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.832+0000 7f10db7fe700 1 -- 172.21.15.112:0/3966922558 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d0011a20 con 0x7f10dc109f60 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3966922558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 msgr2=0x7f10dc10a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 --2- 172.21.15.112:0/3966922558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc10a370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f10d00098a0 tx=0x7f10d0009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3966922558 shutdown_connections 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 --2- 172.21.15.112:0/3966922558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc10a370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3966922558 >> 172.21.15.112:0/3966922558 conn(0x7f10dc103840 msgr2=0x7f10dc105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3966922558 shutdown_connections 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3966922558 wait complete. 2024-04-25T16:29:10.668 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.833+0000 7f10e2a6d700 1 Processor -- start 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e2a6d700 1 -- start start 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e2a6d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e2a6d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc10cec0 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e0809700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e0809700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56634/0 (socket says 172.21.15.112:56634) 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e0809700 1 -- 172.21.15.112:0/3254496035 learned_addr learned my addr 172.21.15.112:0/3254496035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e0809700 1 -- 172.21.15.112:0/3254496035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc19d220 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.834+0000 7f10e0809700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc19cce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10d0011a20 tx=0x7f10d0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.835+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d0003af0 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.835+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d001b8f0 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.835+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc19d4b0 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.835+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10d0023890 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.835+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1a00c0 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.836+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7f10d002c3f0 con 0x7f10dc109f60 2024-04-25T16:29:10.669 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.836+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc064cf0 con 0x7f10dc109f60 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.836+0000 7f10d9ffb700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 0x7f10cc03fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.836+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10d0053c00 con 0x7f10dc109f60 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.836+0000 7f10dbfff700 1 -- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 msgr2=0x7f10cc03fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.836+0000 7f10dbfff700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 0x7f10cc03fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:09.841+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10d001f030 con 0x7f10dc109f60 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.037+0000 7f10dbfff700 1 -- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 msgr2=0x7f10cc03fba0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.037+0000 7f10dbfff700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 0x7f10cc03fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.099+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f10dc1a0290 con 0x7f10dc109f60 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.100+0000 7f10d9ffb700 1 -- 172.21.15.112:0/3254496035 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f10dc1a0290 con 0x7f10dc109f60 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 msgr2=0x7f10cc03fba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 0x7f10cc03fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:10.670 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 msgr2=0x7f10dc19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc19cce0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f10d0011a20 tx=0x7f10d0011b00 comp rx=0 tx=0).stop 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 shutdown_connections 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7f10cc03d690 0x7f10cc03fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 --2- 172.21.15.112:0/3254496035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc109f60 0x7f10dc19cce0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 >> 172.21.15.112:0/3254496035 conn(0x7f10dc103840 msgr2=0x7f10dc103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 shutdown_connections 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:10.102+0000 7f10e2a6d700 1 -- 172.21.15.112:0/3254496035 wait complete. 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for the mgr to restart... 2024-04-25T16:29:10.671 INFO:teuthology.orchestra.run.smithi112.stdout:Waiting for mgr epoch 9... 2024-04-25T16:29:10.919 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:10.843+0000 7fb33f4f7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:29:11.346 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:10.940+0000 7fb33f4f7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:29:11.347 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:11.150+0000 7fb33f4f7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:29:11.763 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:11.347+0000 7fb33f4f7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:29:12.096 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:11.941+0000 7fb33f4f7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:29:12.664 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:12.366+0000 7fb33f4f7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:29:12.665 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:12.462+0000 7fb33f4f7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:29:12.665 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:12.664+0000 7fb33f4f7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:29:13.527 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:13.183+0000 7fb33f4f7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:29:13.527 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:13.276+0000 7fb33f4f7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:29:13.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:13.527+0000 7fb33f4f7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:29:13.847 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:13.659+0000 7fb33f4f7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:29:13.847 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:13.755+0000 7fb33f4f7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:29:14.165 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:13.872+0000 7fb33f4f7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:29:14.596 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:14.168+0000 7fb33f4f7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:29:15.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:14.766+0000 7fb33f4f7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:29:15.038 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:29:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:29:14.982+0000 7fb33f4f7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:29:15.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: Active manager daemon x restarted 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: Activating manager daemon x 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: mgrmap e10: x(active, starting, since 0.00397637s) 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T16:29:15.742 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T16:29:15.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T16:29:15.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: Manager daemon x is now available 2024-04-25T16:29:15.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:15.743 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T16:29:16.200 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T16:29:16.201 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:16 smithi112 ceph-mon[29122]: mgrmap e11: x(active, since 1.00702s) 2024-04-25T16:29:16.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout { 2024-04-25T16:29:16.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T16:29:16.769 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout } 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.155+0000 7fd6204a1700 1 Processor -- start 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd6204a1700 1 -- start start 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd6204a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd618108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd6204a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd618109420 con 0x7fd618108a20 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd61e23d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd618108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd61e23d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd618108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56644/0 (socket says 172.21.15.112:56644) 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd61e23d700 1 -- 172.21.15.112:0/2377870656 learned_addr learned my addr 172.21.15.112:0/2377870656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.156+0000 7fd61e23d700 1 -- 172.21.15.112:0/2377870656 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd618109b50 con 0x7fd618108a20 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.157+0000 7fd61e23d700 1 --2- 172.21.15.112:0/2377870656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd618108e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6140098a0 tx=0x7fd614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe364b47445d8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.157+0000 7fd61d23b700 1 -- 172.21.15.112:0/2377870656 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd614004160 con 0x7fd618108a20 2024-04-25T16:29:16.770 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.157+0000 7fd61d23b700 1 -- 172.21.15.112:0/2377870656 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd614004300 con 0x7fd618108a20 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.157+0000 7fd61d23b700 1 -- 172.21.15.112:0/2377870656 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd614011a20 con 0x7fd618108a20 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 -- 172.21.15.112:0/2377870656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 msgr2=0x7fd618108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 --2- 172.21.15.112:0/2377870656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd618108e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd6140098a0 tx=0x7fd614009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 -- 172.21.15.112:0/2377870656 shutdown_connections 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 --2- 172.21.15.112:0/2377870656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd618108e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 -- 172.21.15.112:0/2377870656 >> 172.21.15.112:0/2377870656 conn(0x7fd6181040e0 msgr2=0x7fd618106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 -- 172.21.15.112:0/2377870656 shutdown_connections 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.158+0000 7fd6204a1700 1 -- 172.21.15.112:0/2377870656 wait complete. 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd6204a1700 1 Processor -- start 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd6204a1700 1 -- start start 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd6204a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd61807c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd6204a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd61810cf00 con 0x7fd618108a20 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd61e23d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd61807c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:16.771 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd61e23d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd61807c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56650/0 (socket says 172.21.15.112:56650) 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd61e23d700 1 -- 172.21.15.112:0/988149473 learned_addr learned my addr 172.21.15.112:0/988149473 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.159+0000 7fd61e23d700 1 -- 172.21.15.112:0/988149473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61819e990 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.160+0000 7fd61e23d700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd61807c950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd614003e70 tx=0x7fd614011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.160+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd614003aa0 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.160+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd61401a8e0 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.160+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61807ce90 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.160+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd614024870 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.160+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6181a3250 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.161+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 9) ==== 49529+0+0 (secure 0 0 0) 0x7fd614022020 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.161+0000 7fd60f7fe700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.161+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 --> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd6040402a0 con 0x7fd60403d680 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.161+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd614052700 con 0x7fd618108a20 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.161+0000 7fd61da3c700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 msgr2=0x7fd60403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:16.772 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.161+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.362+0000 7fd61da3c700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 msgr2=0x7fd60403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.362+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.763+0000 7fd61da3c700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 msgr2=0x7fd60403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:11.763+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:12.564+0000 7fd61da3c700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 msgr2=0x7fd60403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:12.564+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:14.165+0000 7fd61da3c700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 msgr2=0x7fd60403fb90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/108124461 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:14.165+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:14.986+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mgrmap(e 10) ==== 49296+0+0 (secure 0 0 0) 0x7fd61402d3f0 con 0x7fd618108a20 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:14.986+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 msgr2=0x7fd60403fb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:14.986+0000 7fd60f7fe700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.988+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 11) ==== 49423+0+0 (secure 0 0 0) 0x7fd6140547e0 con 0x7fd618108a20 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.988+0000 7fd60f7fe700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd614052290 0x7fd604042fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:16.773 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.988+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd61402be50 con 0x7fd614052290 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.989+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd614052290 0x7fd604042fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.990+0000 7fd61da3c700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd614052290 0x7fd604042fd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd608003a10 tx=0x7fd6080092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.990+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd61402be50 con 0x7fd614052290 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.999+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd618064cf0 con 0x7fd614052290 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:15.999+0000 7fd60f7fe700 1 -- 172.21.15.112:0/988149473 <== mgr.14152 v2:172.21.15.112:6800/3947590362 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd618064cf0 con 0x7fd614052290 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd614052290 msgr2=0x7fd604042fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd614052290 0x7fd604042fd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd608003a10 tx=0x7fd6080092b0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 msgr2=0x7fd61807c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd61807c950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd614003e70 tx=0x7fd614011aa0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 shutdown_connections 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd614052290 0x7fd604042fd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd618108a20 0x7fd61807c950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 --2- 172.21.15.112:0/988149473 >> [v2:172.21.15.112:6800/108124461,v1:172.21.15.112:6801/108124461] conn(0x7fd60403d680 0x7fd60403fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:16.774 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 >> 172.21.15.112:0/988149473 conn(0x7fd6181040e0 msgr2=0x7fd6181058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:16.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 shutdown_connections 2024-04-25T16:29:16.775 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:16.000+0000 7fd6204a1700 1 -- 172.21.15.112:0/988149473 wait complete. 2024-04-25T16:29:16.775 INFO:teuthology.orchestra.run.smithi112.stdout:mgr epoch 9 is available 2024-04-25T16:29:16.775 INFO:teuthology.orchestra.run.smithi112.stdout:Generating a dashboard self-signed certificate... 2024-04-25T16:29:17.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:17 smithi112 ceph-mon[29122]: [25/Apr/2024:16:29:15] ENGINE Bus STARTING 2024-04-25T16:29:17.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:17 smithi112 ceph-mon[29122]: [25/Apr/2024:16:29:15] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T16:29:17.212 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:17 smithi112 ceph-mon[29122]: [25/Apr/2024:16:29:15] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T16:29:17.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:17 smithi112 ceph-mon[29122]: [25/Apr/2024:16:29:15] ENGINE Bus STARTED 2024-04-25T16:29:17.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:17 smithi112 ceph-mon[29122]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T16:29:17.213 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:17 smithi112 ceph-mon[29122]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T16:29:18.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:18 smithi112 ceph-mon[29122]: mgrmap e12: x(active, since 2s) 2024-04-25T16:29:18.273 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:18.274 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:18.274 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:18.281 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.236+0000 7fb99cdd3700 1 Processor -- start 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.236+0000 7fb99cdd3700 1 -- start start 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99cdd3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb9981085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99cdd3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb998108ba0 con 0x7fb9981081a0 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb9981085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb9981085b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55652/0 (socket says 172.21.15.112:55652) 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99659c700 1 -- 172.21.15.112:0/3507674810 learned_addr learned my addr 172.21.15.112:0/3507674810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99659c700 1 -- 172.21.15.112:0/3507674810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9981092d0 con 0x7fb9981081a0 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99659c700 1 --2- 172.21.15.112:0/3507674810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb9981085b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00bb40 tx=0x7fb98c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92eda29a131ab59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99559a700 1 -- 172.21.15.112:0/3507674810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c00dad0 con 0x7fb9981081a0 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99559a700 1 -- 172.21.15.112:0/3507674810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb98c0044e0 con 0x7fb9981081a0 2024-04-25T16:29:18.282 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.237+0000 7fb99559a700 1 -- 172.21.15.112:0/3507674810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c003da0 con 0x7fb9981081a0 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.238+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3507674810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 msgr2=0x7fb9981085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.238+0000 7fb99cdd3700 1 --2- 172.21.15.112:0/3507674810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb9981085b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00bb40 tx=0x7fb98c00be50 comp rx=0 tx=0).stop 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.238+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3507674810 shutdown_connections 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.238+0000 7fb99cdd3700 1 --2- 172.21.15.112:0/3507674810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb9981085b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.238+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3507674810 >> 172.21.15.112:0/3507674810 conn(0x7fb998103840 msgr2=0x7fb998105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.239+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3507674810 shutdown_connections 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.239+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3507674810 wait complete. 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.239+0000 7fb99cdd3700 1 Processor -- start 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99cdd3700 1 -- start start 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99cdd3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb99819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99cdd3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb998109aa0 con 0x7fb9981081a0 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb99819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:18.283 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb99819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55668/0 (socket says 172.21.15.112:55668) 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99659c700 1 -- 172.21.15.112:0/3876668072 learned_addr learned my addr 172.21.15.112:0/3876668072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99659c700 1 -- 172.21.15.112:0/3876668072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99819c120 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb99659c700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb99819bbe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb98c019040 tx=0x7fb98c003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.240+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c017500 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.241+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb98c017b50 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.241+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98c021b00 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.241+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99819c3b0 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.241+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9981a33d0 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.242+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7fb98c02a400 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.242+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb998064cf0 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.242+0000 7fb9877fe700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb98003d6e0 0x7fb98003fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.242+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb98c052e40 con 0x7fb9981081a0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.242+0000 7fb995d9b700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb98003d6e0 0x7fb98003fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:18.284 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.243+0000 7fb995d9b700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb98003d6e0 0x7fb98003fbf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb988006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.248+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb98c012070 con 0x7fb9981081a0 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.463+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb998064cf0 con 0x7fb98003d6e0 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.689+0000 7fb9877fe700 1 -- 172.21.15.112:0/3876668072 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb998064cf0 con 0x7fb98003d6e0 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb98003d6e0 msgr2=0x7fb98003fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb98003d6e0 0x7fb98003fbf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb988006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 msgr2=0x7fb99819bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb99819bbe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb98c019040 tx=0x7fb98c003ea0 comp rx=0 tx=0).stop 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 shutdown_connections 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb98003d6e0 0x7fb98003fbf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 --2- 172.21.15.112:0/3876668072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9981081a0 0x7fb99819bbe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.691+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 >> 172.21.15.112:0/3876668072 conn(0x7fb998103840 msgr2=0x7fb99818fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.692+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 shutdown_connections 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:17.692+0000 7fb99cdd3700 1 -- 172.21.15.112:0/3876668072 wait complete. 2024-04-25T16:29:18.285 INFO:teuthology.orchestra.run.smithi112.stdout:Creating initial admin user... 2024-04-25T16:29:19.328 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:19 smithi112 ceph-mon[29122]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:19.329 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:19.839 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$cehd0K/OHuG4CmtWRTtS0eDPr9sLzIlCF5qclVVQHjY04FyP8bQVq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714062559, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.776+0000 7f18bca7a700 1 Processor -- start 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.776+0000 7f18bca7a700 1 -- start start 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18bca7a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b8108e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18bca7a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8109420 con 0x7f18b8108a20 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b8108e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b8108e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55676/0 (socket says 172.21.15.112:55676) 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18b659c700 1 -- 172.21.15.112:0/3251604483 learned_addr learned my addr 172.21.15.112:0/3251604483 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18b659c700 1 -- 172.21.15.112:0/3251604483 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b8109b50 con 0x7f18b8108a20 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18b659c700 1 --2- 172.21.15.112:0/3251604483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b8108e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098a0 tx=0x7f18a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d85937d4353b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.777+0000 7f18b559a700 1 -- 172.21.15.112:0/3251604483 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8004160 con 0x7f18b8108a20 2024-04-25T16:29:19.840 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18b559a700 1 -- 172.21.15.112:0/3251604483 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a8004300 con 0x7f18b8108a20 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18b559a700 1 -- 172.21.15.112:0/3251604483 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8011a20 con 0x7f18b8108a20 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18bca7a700 1 -- 172.21.15.112:0/3251604483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 msgr2=0x7f18b8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18bca7a700 1 --2- 172.21.15.112:0/3251604483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b8108e30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18a80098a0 tx=0x7f18a8009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18bca7a700 1 -- 172.21.15.112:0/3251604483 shutdown_connections 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18bca7a700 1 --2- 172.21.15.112:0/3251604483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b8108e30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.778+0000 7f18bca7a700 1 -- 172.21.15.112:0/3251604483 >> 172.21.15.112:0/3251604483 conn(0x7f18b81040e0 msgr2=0x7f18b8106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.779+0000 7f18bca7a700 1 -- 172.21.15.112:0/3251604483 shutdown_connections 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.779+0000 7f18bca7a700 1 -- 172.21.15.112:0/3251604483 wait complete. 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.779+0000 7f18bca7a700 1 Processor -- start 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.779+0000 7f18bca7a700 1 -- start start 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18bca7a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b807c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18bca7a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f18b810cf00 con 0x7f18b8108a20 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b807c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18b659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b807c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55684/0 (socket says 172.21.15.112:55684) 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18b659c700 1 -- 172.21.15.112:0/4258861376 learned_addr learned my addr 172.21.15.112:0/4258861376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:19.841 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18b659c700 1 -- 172.21.15.112:0/4258861376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b807ceb0 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18b659c700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b807c970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18a8003e70 tx=0x7f18a8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8003aa0 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a801a8e0 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b81a0b90 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.780+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a8024870 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.781+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a1010 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.782+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b8064cf0 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.782+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f18a8022020 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.782+0000 7f18a77fe700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f18a003d680 0x7f18a003fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.782+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f18a8052800 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.782+0000 7f18b5d9b700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f18a003d680 0x7f18a003fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.783+0000 7f18b5d9b700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f18a003d680 0x7f18a003fb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:18.787+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f18a8029df0 con 0x7f18b8108a20 2024-04-25T16:29:19.842 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.010+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f18b8105fb0 con 0x7f18a003d680 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.236+0000 7f18a77fe700 1 -- 172.21.15.112:0/4258861376 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f18b8105fb0 con 0x7f18a003d680 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f18a003d680 msgr2=0x7f18a003fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f18a003d680 0x7f18a003fb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18ac006fd0 tx=0x7f18ac006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 msgr2=0x7f18b807c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b807c970 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18a8003e70 tx=0x7f18a8011aa0 comp rx=0 tx=0).stop 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 shutdown_connections 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f18a003d680 0x7f18a003fb90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 --2- 172.21.15.112:0/4258861376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18b8108a20 0x7f18b807c970 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 >> 172.21.15.112:0/4258861376 conn(0x7f18b81040e0 msgr2=0x7f18b81058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 shutdown_connections 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:19.238+0000 7f18bca7a700 1 -- 172.21.15.112:0/4258861376 wait complete. 2024-04-25T16:29:19.843 INFO:teuthology.orchestra.run.smithi112.stdout:Fetching dashboard port number... 2024-04-25T16:29:20.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:20 smithi112 ceph-mon[29122]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:20.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:20 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T16:29:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.254+0000 7f984c06d700 1 Processor -- start 2024-04-25T16:29:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.254+0000 7f984c06d700 1 -- start start 2024-04-25T16:29:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f984c06d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f984410c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f984c06d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f984410ccb0 con 0x7f984407ccb0 2024-04-25T16:29:21.181 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f9849e09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f984410c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f9849e09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f984410c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55692/0 (socket says 172.21.15.112:55692) 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f9849e09700 1 -- 172.21.15.112:0/714680166 learned_addr learned my addr 172.21.15.112:0/714680166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f9849e09700 1 -- 172.21.15.112:0/714680166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984410ce30 con 0x7f984407ccb0 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f9849e09700 1 --2- 172.21.15.112:0/714680166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f984410c6c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f98400098a0 tx=0x7f9840009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388dc873e6900ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.255+0000 7f9848e07700 1 -- 172.21.15.112:0/714680166 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9840004160 con 0x7f984407ccb0 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f9848e07700 1 -- 172.21.15.112:0/714680166 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9840004300 con 0x7f984407ccb0 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f9848e07700 1 -- 172.21.15.112:0/714680166 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9840011a20 con 0x7f984407ccb0 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f984c06d700 1 -- 172.21.15.112:0/714680166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 msgr2=0x7f984410c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f984c06d700 1 --2- 172.21.15.112:0/714680166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f984410c6c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f98400098a0 tx=0x7f9840009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f984c06d700 1 -- 172.21.15.112:0/714680166 shutdown_connections 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f984c06d700 1 --2- 172.21.15.112:0/714680166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f984410c6c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.256+0000 7f984c06d700 1 -- 172.21.15.112:0/714680166 >> 172.21.15.112:0/714680166 conn(0x7f9844104130 msgr2=0x7f9844106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:21.182 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.257+0000 7f984c06d700 1 -- 172.21.15.112:0/714680166 shutdown_connections 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.257+0000 7f984c06d700 1 -- 172.21.15.112:0/714680166 wait complete. 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.257+0000 7f984c06d700 1 Processor -- start 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.257+0000 7f984c06d700 1 -- start start 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.257+0000 7f984c06d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f98441a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.257+0000 7f984c06d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f984410def0 con 0x7f984407ccb0 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f9849e09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f98441a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f9849e09700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f98441a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55704/0 (socket says 172.21.15.112:55704) 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f9849e09700 1 -- 172.21.15.112:0/3356941939 learned_addr learned my addr 172.21.15.112:0/3356941939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f9849e09700 1 -- 172.21.15.112:0/3356941939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98441a0580 con 0x7f984407ccb0 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f9849e09700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f98441a0040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98400119f0 tx=0x7f9840011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98400039e0 con 0x7f984407ccb0 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f984001b8f0 con 0x7f984407ccb0 2024-04-25T16:29:21.183 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441a0810 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98400237a0 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.258+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a3420 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.259+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9844064cf0 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.259+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 12) ==== 49529+0+0 (secure 0 0 0) 0x7f9840023aa0 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.260+0000 7f983affd700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f983003d630 0x7f983003fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.260+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9840052d80 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.260+0000 7f9849608700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f983003d630 0x7f983003fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.260+0000 7f9849608700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f983003d630 0x7f983003fb40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9834006fd0 tx=0x7f9834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.265+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f984001f030 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.468+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f98441a35f0 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.468+0000 7f983affd700 1 -- 172.21.15.112:0/3356941939 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f9840019e20 con 0x7f984407ccb0 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f983003d630 msgr2=0x7f983003fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f983003d630 0x7f983003fb40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9834006fd0 tx=0x7f9834006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:21.184 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 msgr2=0x7f98441a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f98441a0040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f98400119f0 tx=0x7f9840011ad0 comp rx=0 tx=0).stop 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 shutdown_connections 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f983003d630 0x7f983003fb40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 --2- 172.21.15.112:0/3356941939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f984407ccb0 0x7f98441a0040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 >> 172.21.15.112:0/3356941939 conn(0x7f9844104130 msgr2=0x7f98441085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 shutdown_connections 2024-04-25T16:29:21.185 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:20.470+0000 7f984c06d700 1 -- 172.21.15.112:0/3356941939 wait complete. 2024-04-25T16:29:21.187 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T16:29:21.188 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout disabled 2024-04-25T16:29:21.194 INFO:teuthology.orchestra.run.smithi112.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T16:29:21.194 INFO:teuthology.orchestra.run.smithi112.stdout:systemctl: stdout inactive 2024-04-25T16:29:21.194 INFO:teuthology.orchestra.run.smithi112.stdout:firewalld.service is not enabled 2024-04-25T16:29:21.194 INFO:teuthology.orchestra.run.smithi112.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout:Ceph Dashboard is now available at: 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: URL: https://smithi112:8443/ 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: User: admin 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: Password: 7j85320gp0 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:21.196 INFO:teuthology.orchestra.run.smithi112.stdout:Saving cluster configuration to /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config directory 2024-04-25T16:29:21.198 INFO:teuthology.orchestra.run.smithi112.stdout:Enabling autotune for osd_memory_target 2024-04-25T16:29:21.308 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:21 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3356941939' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T16:29:22.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:22 smithi112 ceph-mon[29122]: mgrmap e13: x(active, since 6s) 2024-04-25T16:29:22.639 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f513d92d700 1 Processor -- start 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f513d92d700 1 -- start start 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f513d92d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513810c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f513d92d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f513810cc80 con 0x7f513807cc70 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513810c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513810c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55714/0 (socket says 172.21.15.112:55714) 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f5136ffd700 1 -- 172.21.15.112:0/89625734 learned_addr learned my addr 172.21.15.112:0/89625734 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.614+0000 7f5136ffd700 1 -- 172.21.15.112:0/89625734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513810ce00 con 0x7f513807cc70 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f5136ffd700 1 --2- 172.21.15.112:0/89625734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513810c6b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51280098a0 tx=0x7f5128009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044f3a7fc9fc617 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f5135ffb700 1 -- 172.21.15.112:0/89625734 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5128004160 con 0x7f513807cc70 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f5135ffb700 1 -- 172.21.15.112:0/89625734 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5128004300 con 0x7f513807cc70 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f5135ffb700 1 -- 172.21.15.112:0/89625734 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5128011a20 con 0x7f513807cc70 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 -- 172.21.15.112:0/89625734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 msgr2=0x7f513810c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:22.640 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 --2- 172.21.15.112:0/89625734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513810c6b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f51280098a0 tx=0x7f5128009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 -- 172.21.15.112:0/89625734 shutdown_connections 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 --2- 172.21.15.112:0/89625734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513810c6b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 -- 172.21.15.112:0/89625734 >> 172.21.15.112:0/89625734 conn(0x7f51381040e0 msgr2=0x7f5138106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 -- 172.21.15.112:0/89625734 shutdown_connections 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.615+0000 7f513d92d700 1 -- 172.21.15.112:0/89625734 wait complete. 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.616+0000 7f513d92d700 1 Processor -- start 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.616+0000 7f513d92d700 1 -- start start 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.616+0000 7f513d92d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.616+0000 7f513d92d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f513810dec0 con 0x7f513807cc70 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f5136ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55718/0 (socket says 172.21.15.112:55718) 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f5136ffd700 1 -- 172.21.15.112:0/128812788 learned_addr learned my addr 172.21.15.112:0/128812788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f5136ffd700 1 -- 172.21.15.112:0/128812788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513819e310 con 0x7f513807cc70 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f5136ffd700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513819ddd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51280119f0 tx=0x7f5128011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:22.641 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51280039e0 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512801b8f0 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513819e5a0 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51280237a0 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.617+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51381a1170 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.618+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f5128023a50 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.618+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5138064cf0 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.618+0000 7f513c92b700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f512003d730 0x7f512003fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.618+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5128052f00 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.619+0000 7f51367fc700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f512003d730 0x7f512003fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.619+0000 7f51367fc700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f512003d730 0x7f512003fc40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f512c006fd0 tx=0x7f512c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.624+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f512801f030 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.824+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f51381a1870 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.825+0000 7f513c92b700 1 -- 172.21.15.112:0/128812788 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f512801baa0 con 0x7f513807cc70 2024-04-25T16:29:22.642 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f512003d730 msgr2=0x7f512003fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f512003d730 0x7f512003fc40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f512c006fd0 tx=0x7f512c006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 msgr2=0x7f513819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513819ddd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51280119f0 tx=0x7f5128011ad0 comp rx=0 tx=0).stop 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 shutdown_connections 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f512003d730 0x7f512003fc40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 --2- 172.21.15.112:0/128812788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f513807cc70 0x7f513819ddd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 >> 172.21.15.112:0/128812788 conn(0x7f51381040e0 msgr2=0x7f5138105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 shutdown_connections 2024-04-25T16:29:22.643 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:21.827+0000 7f513d92d700 1 -- 172.21.15.112:0/128812788 wait complete. 2024-04-25T16:29:23.989 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd190b98700 1 Processor -- start 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd190b98700 1 -- start start 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd190b98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c10c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd190b98700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10ccd0 con 0x7fd18c07d520 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd18a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c10c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd18a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c10c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55726/0 (socket says 172.21.15.112:55726) 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.083+0000 7fd18a59c700 1 -- 172.21.15.112:0/3523923357 learned_addr learned my addr 172.21.15.112:0/3523923357 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.084+0000 7fd18a59c700 1 -- 172.21.15.112:0/3523923357 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c10ce50 con 0x7fd18c07d520 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.084+0000 7fd18a59c700 1 --2- 172.21.15.112:0/3523923357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c10c700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0098a0 tx=0x7fd17c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa7303c5406aef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.084+0000 7fd18959a700 1 -- 172.21.15.112:0/3523923357 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17c004160 con 0x7fd18c07d520 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.084+0000 7fd18959a700 1 -- 172.21.15.112:0/3523923357 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd17c004300 con 0x7fd18c07d520 2024-04-25T16:29:23.990 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.084+0000 7fd18959a700 1 -- 172.21.15.112:0/3523923357 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17c011a20 con 0x7fd18c07d520 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 -- 172.21.15.112:0/3523923357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 msgr2=0x7fd18c10c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 --2- 172.21.15.112:0/3523923357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c10c700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0098a0 tx=0x7fd17c009bb0 comp rx=0 tx=0).stop 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 -- 172.21.15.112:0/3523923357 shutdown_connections 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 --2- 172.21.15.112:0/3523923357 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c10c700 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 -- 172.21.15.112:0/3523923357 >> 172.21.15.112:0/3523923357 conn(0x7fd18c104110 msgr2=0x7fd18c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 -- 172.21.15.112:0/3523923357 shutdown_connections 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.085+0000 7fd190b98700 1 -- 172.21.15.112:0/3523923357 wait complete. 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd190b98700 1 Processor -- start 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd190b98700 1 -- start start 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd190b98700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd190b98700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c10df10 con 0x7fd18c07d520 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd18a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd18a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55738/0 (socket says 172.21.15.112:55738) 2024-04-25T16:29:23.991 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd18a59c700 1 -- 172.21.15.112:0/477832871 learned_addr learned my addr 172.21.15.112:0/477832871 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd18a59c700 1 -- 172.21.15.112:0/477832871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c19e320 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.086+0000 7fd18a59c700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c19dde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd17c006af0 tx=0x7fd17c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.087+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17c003b20 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.087+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd17c01b8f0 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.087+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c19e5b0 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.087+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd17c0238c0 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.087+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c1a1180 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.088+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fd17c023a60 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.088+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd18c064cf0 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.088+0000 7fd17b7fe700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd17403d730 0x7fd17403fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.088+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd17c052f80 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.088+0000 7fd189d9b700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd17403d730 0x7fd17403fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.089+0000 7fd189d9b700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd17403d730 0x7fd17403fc40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd180006fd0 tx=0x7fd180006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.093+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd17c01f030 con 0x7fd18c07d520 2024-04-25T16:29:23.992 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.401+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd18c1a1820 con 0x7fd18c07d520 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.404+0000 7fd17b7fe700 1 -- 172.21.15.112:0/477832871 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fd17c01baa0 con 0x7fd18c07d520 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd17403d730 msgr2=0x7fd17403fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd17403d730 0x7fd17403fc40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd180006fd0 tx=0x7fd180006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 msgr2=0x7fd18c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c19dde0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd17c006af0 tx=0x7fd17c011aa0 comp rx=0 tx=0).stop 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 shutdown_connections 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd17403d730 0x7fd17403fc40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 --2- 172.21.15.112:0/477832871 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd18c07d520 0x7fd18c19dde0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 >> 172.21.15.112:0/477832871 conn(0x7fd18c104110 msgr2=0x7fd18c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 shutdown_connections 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr 2024-04-25T16:29:23.406+0000 7fd190b98700 1 -- 172.21.15.112:0/477832871 wait complete. 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T16:29:23.993 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: ceph telemetry on 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout:For more information see: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T16:29:23.994 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:23.995 INFO:teuthology.orchestra.run.smithi112.stdout:Bootstrap complete. 2024-04-25T16:29:24.029 INFO:tasks.cephadm:Fetching config... 2024-04-25T16:29:24.029 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:29:24.029 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T16:29:24.044 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T16:29:24.045 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:29:24.045 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T16:29:24.100 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T16:29:24.100 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:29:24.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T16:29:24.164 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T16:29:24.164 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:29:24.164 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T16:29:24.221 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T16:29:24.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T16:29:24.292 INFO:teuthology.orchestra.run.smithi112.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:29:24.302 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T16:29:24.341 INFO:teuthology.orchestra.run.smithi116.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:29:24.353 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T16:29:24.391 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnsQh7Rl9YmrqDuoSvUaIro2zSEapoIh27Qal8Ff/NH2Z52gZUqemEtV0DIePEgDQ3mcOBRisI5PJr97Ft+HYJcXO8/DUcLGZNySiPu44bwhUROWUJHKxmi5ohjfyV98qk5QsQKxxvQhzJdmgt9yZKKeWg0e7buTmu0i5LT1fdzgC9+8MC3b/AA4IyC8vsGW2S1p0drF6cznTrWoTVeMzCRDR3Xd2zTB9FqC/Oysry51H79hKlPUBfsNrKVDY7eLkojHriKguiQ8XdrKnUt5TUOTAAGLoxiCkhBfjI4mJ3WHiBVeJHaHyKNdgGlCl66tP85WHHYIApWkOZFiPmYnZYyN24BtbohOcOD27eewdI/AmYNF6Sahw8PEowy/JL1k/nouvIAn9JuOZmvCEppUtHw9dpud8HaH+CUOH3jrIYJ9qU0YuHmwcpf1uPG7tloJLg3tA4K4xM3xDDmvc/3yNNy4OqfELAQgBRbVgWKN4yrsWa1s1v8dosJ3hDOZ7MKw8= ceph-a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:29:24.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T16:29:24.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:24 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/477832871' entity='client.admin' 2024-04-25T16:29:24.637 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:27.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 -- 172.21.15.112:0/541531954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 msgr2=0x7f48b81069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:27.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 --2- 172.21.15.112:0/541531954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b81069c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009910 tx=0x7f48a8009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:27.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 -- 172.21.15.112:0/541531954 shutdown_connections 2024-04-25T16:29:27.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 --2- 172.21.15.112:0/541531954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b81069c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:27.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 -- 172.21.15.112:0/541531954 >> 172.21.15.112:0/541531954 conn(0x7f48b80fe000 msgr2=0x7f48b8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 -- 172.21.15.112:0/541531954 shutdown_connections 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 -- 172.21.15.112:0/541531954 wait complete. 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.152+0000 7f48bd3b1700 1 Processor -- start 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48bd3b1700 1 -- start start 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48bd3b1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b8103d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48bd3b1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48b8109a60 con 0x7f48b81045a0 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48b6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b8103d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48b6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b8103d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35978/0 (socket says 172.21.15.112:35978) 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48b6ffd700 1 -- 172.21.15.112:0/3823869542 learned_addr learned my addr 172.21.15.112:0/3823869542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.153+0000 7f48b6ffd700 1 -- 172.21.15.112:0/3823869542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b8104260 con 0x7f48b81045a0 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.154+0000 7f48b6ffd700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b8103d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48a80050d0 tx=0x7f48a8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.154+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a8027070 con 0x7f48b81045a0 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.154+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48a800be10 con 0x7f48b81045a0 2024-04-25T16:29:27.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.154+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b81024b0 con 0x7f48b81045a0 2024-04-25T16:29:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.154+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f48a8022d80 con 0x7f48b81045a0 2024-04-25T16:29:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.154+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b8102990 con 0x7f48b81045a0 2024-04-25T16:29:27.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.155+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f48a802a070 con 0x7f48b81045a0 2024-04-25T16:29:27.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.155+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b810a1e0 con 0x7f48b81045a0 2024-04-25T16:29:27.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.155+0000 7f489ffff700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48a003d5c0 0x7f48a003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:27.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.155+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f48a801e030 con 0x7f48b81045a0 2024-04-25T16:29:27.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.155+0000 7f48b67fc700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48a003d5c0 0x7f48a003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:27.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.156+0000 7f48b67fc700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48a003d5c0 0x7f48a003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006fd0 tx=0x7f48ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:27.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.161+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48a8013070 con 0x7f48b81045a0 2024-04-25T16:29:27.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.365+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f48b8192400 con 0x7f48b81045a0 2024-04-25T16:29:27.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.368+0000 7f489ffff700 1 -- 172.21.15.112:0/3823869542 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f48a80292c0 con 0x7f48b81045a0 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48a003d5c0 msgr2=0x7f48a003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48a003d5c0 0x7f48a003fad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48ac006fd0 tx=0x7f48ac006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 msgr2=0x7f48b8103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b8103d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48a80050d0 tx=0x7f48a8009f80 comp rx=0 tx=0).stop 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 shutdown_connections 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48a003d5c0 0x7f48a003fad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 --2- 172.21.15.112:0/3823869542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48b81045a0 0x7f48b8103d20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 >> 172.21.15.112:0/3823869542 conn(0x7f48b80fe000 msgr2=0x7f48b80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 shutdown_connections 2024-04-25T16:29:27.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:27.371+0000 7f48bd3b1700 1 -- 172.21.15.112:0/3823869542 wait complete. 2024-04-25T16:29:27.969 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T16:29:27.969 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T16:29:28.533 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:28.656 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:28 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3823869542' entity='client.admin' 2024-04-25T16:29:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi112", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:29:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 -- 172.21.15.112:0/1185397493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 msgr2=0x7fa964106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 --2- 172.21.15.112:0/1185397493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964106a00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa960009910 tx=0x7fa960009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:30.079 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 -- 172.21.15.112:0/1185397493 shutdown_connections 2024-04-25T16:29:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 --2- 172.21.15.112:0/1185397493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964106a00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 -- 172.21.15.112:0/1185397493 >> 172.21.15.112:0/1185397493 conn(0x7fa9640fe040 msgr2=0x7fa9641004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 -- 172.21.15.112:0/1185397493 shutdown_connections 2024-04-25T16:29:30.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.079+0000 7fa96bbdf700 1 -- 172.21.15.112:0/1185397493 wait complete. 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96bbdf700 1 Processor -- start 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96bbdf700 1 -- start start 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96bbdf700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96bbdf700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa964109aa0 con 0x7fa9641045e0 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96997b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96997b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35986/0 (socket says 172.21.15.112:35986) 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96997b700 1 -- 172.21.15.112:0/3971605737 learned_addr learned my addr 172.21.15.112:0/3971605737 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.080+0000 7fa96997b700 1 -- 172.21.15.112:0/3971605737 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa964199b60 con 0x7fa9641045e0 2024-04-25T16:29:30.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.081+0000 7fa96997b700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964199620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9600050d0 tx=0x7fa960009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.081+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa960027070 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.081+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa96000be10 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.081+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa964199df0 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.081+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa960022d80 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.081+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96419ca00 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.082+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fa96002a070 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.082+0000 7fa95affd700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa95003d5c0 0x7fa95003fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.082+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa96001e030 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.082+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96410a220 con 0x7fa9641045e0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.082+0000 7fa96917a700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa95003d5c0 0x7fa95003fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:30.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.082+0000 7fa96917a700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa95003d5c0 0x7fa95003fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa954006fd0 tx=0x7fa954006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:30.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.087+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa960031070 con 0x7fa9641045e0 2024-04-25T16:29:30.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.286+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa96402e580 con 0x7fa95003d5c0 2024-04-25T16:29:30.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.290+0000 7fa95affd700 1 -- 172.21.15.112:0/3971605737 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa96402e580 con 0x7fa95003d5c0 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa95003d5c0 msgr2=0x7fa95003fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa95003d5c0 0x7fa95003fad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa954006fd0 tx=0x7fa954006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 msgr2=0x7fa964199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964199620 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa9600050d0 tx=0x7fa960009f80 comp rx=0 tx=0).stop 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 shutdown_connections 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa95003d5c0 0x7fa95003fad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:30.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 --2- 172.21.15.112:0/3971605737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa9641045e0 0x7fa964199620 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 >> 172.21.15.112:0/3971605737 conn(0x7fa9640fe040 msgr2=0x7fa9640fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 shutdown_connections 2024-04-25T16:29:30.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:30.292+0000 7fa96bbdf700 1 -- 172.21.15.112:0/3971605737 wait complete. 2024-04-25T16:29:31.047 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi116 2024-04-25T16:29:31.047 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:29:31.047 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/etc/ceph/ceph.conf 2024-04-25T16:29:31.066 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:29:31.067 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:29:31.125 INFO:tasks.cephadm:Adding host smithi116 to orchestrator... 2024-04-25T16:29:31.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch host add smithi116 2024-04-25T16:29:31.374 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 -- 172.21.15.112:0/3042568909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 msgr2=0x7f02b0109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 --2- 172.21.15.112:0/3042568909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0109e00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f02ac009910 tx=0x7f02ac009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 -- 172.21.15.112:0/3042568909 shutdown_connections 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 --2- 172.21.15.112:0/3042568909 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0109e00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 -- 172.21.15.112:0/3042568909 >> 172.21.15.112:0/3042568909 conn(0x7f02b00ff500 msgr2=0x7f02b0101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 -- 172.21.15.112:0/3042568909 shutdown_connections 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.742+0000 7f02b7981700 1 -- 172.21.15.112:0/3042568909 wait complete. 2024-04-25T16:29:32.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b7981700 1 Processor -- start 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b7981700 1 -- start start 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b7981700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b7981700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02b010df40 con 0x7f02b01099f0 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b571d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b571d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0075990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36008/0 (socket says 172.21.15.112:36008) 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b571d700 1 -- 172.21.15.112:0/2389165719 learned_addr learned my addr 172.21.15.112:0/2389165719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b571d700 1 -- 172.21.15.112:0/2389165719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b00777e0 con 0x7f02b01099f0 2024-04-25T16:29:32.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02b571d700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0075990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0056d0 tx=0x7f02ac0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:32.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02ac027070 con 0x7f02b01099f0 2024-04-25T16:29:32.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.743+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02ac00be10 con 0x7f02b01099f0 2024-04-25T16:29:32.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.744+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b0075ed0 con 0x7f02b01099f0 2024-04-25T16:29:32.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.744+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02ac022d20 con 0x7f02b01099f0 2024-04-25T16:29:32.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.744+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b0076390 con 0x7f02b01099f0 2024-04-25T16:29:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.744+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b0105100 con 0x7f02b01099f0 2024-04-25T16:29:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.744+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f02ac02a070 con 0x7f02b01099f0 2024-04-25T16:29:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.745+0000 7f02a67fc700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f029c03d610 0x7f029c03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.745+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f02ac01e030 con 0x7f02b01099f0 2024-04-25T16:29:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.745+0000 7f02b4f1c700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f029c03d610 0x7f029c03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.745+0000 7f02b4f1c700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f029c03d610 0x7f029c03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006fd0 tx=0x7f02a0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:32.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.750+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f02ac013070 con 0x7f02b01099f0 2024-04-25T16:29:32.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:32.951+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi116", "target": ["mon-mgr", ""]}) -- 0x7f02b0076f60 con 0x7f029c03d610 2024-04-25T16:29:33.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:33 smithi112 ceph-mon[29122]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi116", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:34.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:34 smithi112 ceph-mon[29122]: Deploying cephadm binary to smithi116 2024-04-25T16:29:36.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:35 smithi112 ceph-mon[29122]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:37.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.126+0000 7f02a67fc700 1 -- 172.21.15.112:0/2389165719 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f02b0076f60 con 0x7f029c03d610 2024-04-25T16:29:37.126 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi116' with addr '172.21.15.116' 2024-04-25T16:29:37.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f029c03d610 msgr2=0x7f029c03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:37.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f029c03d610 0x7f029c03fb20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f02a0006fd0 tx=0x7f02a0006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 msgr2=0x7f02b0075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0075990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0056d0 tx=0x7f02ac0057b0 comp rx=0 tx=0).stop 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 shutdown_connections 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f029c03d610 0x7f029c03fb20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 --2- 172.21.15.112:0/2389165719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02b01099f0 0x7f02b0075990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 >> 172.21.15.112:0/2389165719 conn(0x7f02b00ff500 msgr2=0x7f02b0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.128+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 shutdown_connections 2024-04-25T16:29:37.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:37.129+0000 7f02b7981700 1 -- 172.21.15.112:0/2389165719 wait complete. 2024-04-25T16:29:37.726 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T16:29:37.957 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:38.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:38 smithi112 ceph-mon[29122]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:38.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:38 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:38.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:38 smithi112 ceph-mon[29122]: Added host smithi116 2024-04-25T16:29:38.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:38 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:38.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:38 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:39.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:39.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:39 smithi112 ceph-mon[29122]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.525+0000 7f2842918700 1 -- 172.21.15.112:0/1825933365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 msgr2=0x7f283c108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.525+0000 7f2842918700 1 --2- 172.21.15.112:0/1825933365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c108d30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f282c009910 tx=0x7f282c009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.525+0000 7f2842918700 1 -- 172.21.15.112:0/1825933365 shutdown_connections 2024-04-25T16:29:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.526+0000 7f2842918700 1 --2- 172.21.15.112:0/1825933365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c108d30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:39.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.526+0000 7f2842918700 1 -- 172.21.15.112:0/1825933365 >> 172.21.15.112:0/1825933365 conn(0x7f283c0fe450 msgr2=0x7f283c1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.526+0000 7f2842918700 1 -- 172.21.15.112:0/1825933365 shutdown_connections 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.526+0000 7f2842918700 1 -- 172.21.15.112:0/1825933365 wait complete. 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.526+0000 7f2842918700 1 Processor -- start 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.526+0000 7f2842918700 1 -- start start 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f2842918700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f2842918700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f283c10ce70 con 0x7f283c108920 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f283bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f283bfff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58160/0 (socket says 172.21.15.112:58160) 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f283bfff700 1 -- 172.21.15.112:0/501236380 learned_addr learned my addr 172.21.15.112:0/501236380 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f283bfff700 1 -- 172.21.15.112:0/501236380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c06c850 con 0x7f283c108920 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f283bfff700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c19a650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f282c0056d0 tx=0x7f282c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c027070 con 0x7f283c108920 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f282c00be10 con 0x7f283c108920 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c19ac50 con 0x7f283c108920 2024-04-25T16:29:39.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f282c022d20 con 0x7f283c108920 2024-04-25T16:29:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.527+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c19b110 con 0x7f283c108920 2024-04-25T16:29:39.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.528+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f282c02a070 con 0x7f283c108920 2024-04-25T16:29:39.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.528+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c104030 con 0x7f283c108920 2024-04-25T16:29:39.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.528+0000 7f28397fa700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f282403d610 0x7f282403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:39.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.529+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f282c01e030 con 0x7f283c108920 2024-04-25T16:29:39.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.529+0000 7f283b7fe700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f282403d610 0x7f282403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:39.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.529+0000 7f283b7fe700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f282403d610 0x7f282403fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2830006fd0 tx=0x7f2830006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:39.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.534+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f282c013070 con 0x7f283c108920 2024-04-25T16:29:39.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.741+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f283c06bfa0 con 0x7f282403d610 2024-04-25T16:29:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.743+0000 7f28397fa700 1 -- 172.21.15.112:0/501236380 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f283c06bfa0 con 0x7f282403d610 2024-04-25T16:29:39.743 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:39.743 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.116", "hostname": "smithi116", "labels": [], "status": ""}] 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f282403d610 msgr2=0x7f282403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f282403d610 0x7f282403fb20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2830006fd0 tx=0x7f2830006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 msgr2=0x7f283c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c19a650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f282c0056d0 tx=0x7f282c0057b0 comp rx=0 tx=0).stop 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 shutdown_connections 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f282403d610 0x7f282403fb20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 --2- 172.21.15.112:0/501236380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f283c108920 0x7f283c19a650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 >> 172.21.15.112:0/501236380 conn(0x7f283c0fe450 msgr2=0x7f283c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 shutdown_connections 2024-04-25T16:29:39.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:39.745+0000 7f2842918700 1 -- 172.21.15.112:0/501236380 wait complete. 2024-04-25T16:29:40.389 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi133 2024-04-25T16:29:40.389 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:29:40.390 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.conf 2024-04-25T16:29:40.409 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:29:40.410 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:29:40.466 INFO:tasks.cephadm:Adding host smithi133 to orchestrator... 2024-04-25T16:29:40.466 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch host add smithi133 2024-04-25T16:29:40.700 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:41.020 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:40 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:41.021 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:40 smithi112 ceph-mon[29122]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:29:41.969 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:41 smithi112 ceph-mon[29122]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 -- 172.21.15.112:0/655314724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 msgr2=0x7fa1b8108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 --2- 172.21.15.112:0/655314724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b8108d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009910 tx=0x7fa1ac009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 -- 172.21.15.112:0/655314724 shutdown_connections 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 --2- 172.21.15.112:0/655314724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b8108d30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 -- 172.21.15.112:0/655314724 >> 172.21.15.112:0/655314724 conn(0x7fa1b80fe450 msgr2=0x7fa1b81008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 -- 172.21.15.112:0/655314724 shutdown_connections 2024-04-25T16:29:42.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.210+0000 7fa1bf020700 1 -- 172.21.15.112:0/655314724 wait complete. 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bf020700 1 Processor -- start 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bf020700 1 -- start start 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bf020700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bf020700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b810ce70 con 0x7fa1b8108920 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bcdbc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bcdbc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58168/0 (socket says 172.21.15.112:58168) 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bcdbc700 1 -- 172.21.15.112:0/2749978517 learned_addr learned my addr 172.21.15.112:0/2749978517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bcdbc700 1 -- 172.21.15.112:0/2749978517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b806c870 con 0x7fa1b8108920 2024-04-25T16:29:42.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1bcdbc700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b819a630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0056d0 tx=0x7fa1ac0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.211+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac027070 con 0x7fa1b8108920 2024-04-25T16:29:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.212+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b819ac30 con 0x7fa1b8108920 2024-04-25T16:29:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.212+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1ac00be10 con 0x7fa1b8108920 2024-04-25T16:29:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.212+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1ac022d20 con 0x7fa1b8108920 2024-04-25T16:29:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.212+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b819b0f0 con 0x7fa1b8108920 2024-04-25T16:29:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.213+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fa1ac02a070 con 0x7fa1b8108920 2024-04-25T16:29:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.213+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b8104030 con 0x7fa1b8108920 2024-04-25T16:29:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.213+0000 7fa1b5ffb700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1a803d610 0x7fa1a803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.213+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa1ac01e030 con 0x7fa1b8108920 2024-04-25T16:29:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.213+0000 7fa1b7fff700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1a803d610 0x7fa1a803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.213+0000 7fa1b7fff700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1a803d610 0x7fa1a803fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4006fd0 tx=0x7fa1a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:42.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.218+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1b806c870 con 0x7fa1b8108920 2024-04-25T16:29:42.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:42.422+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}) -- 0x7fa1b806bfc0 con 0x7fa1a803d610 2024-04-25T16:29:44.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:44.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:43 smithi112 ceph-mon[29122]: Deploying cephadm binary to smithi133 2024-04-25T16:29:45.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:44 smithi112 ceph-mon[29122]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:29:45.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:44 smithi112 ceph-mon[29122]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:29:46.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:45 smithi112 ceph-mon[29122]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:46.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.490+0000 7fa1b5ffb700 1 -- 172.21.15.112:0/2749978517 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa1b806bfc0 con 0x7fa1a803d610 2024-04-25T16:29:46.490 INFO:teuthology.orchestra.run.smithi112.stdout:Added host 'smithi133' with addr '172.21.15.133' 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1a803d610 msgr2=0x7fa1a803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1a803d610 0x7fa1a803fb20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4006fd0 tx=0x7fa1a4006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 msgr2=0x7fa1b819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b819a630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0056d0 tx=0x7fa1ac0057b0 comp rx=0 tx=0).stop 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 shutdown_connections 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1a803d610 0x7fa1a803fb20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:46.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 --2- 172.21.15.112:0/2749978517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1b8108920 0x7fa1b819a630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:46.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 >> 172.21.15.112:0/2749978517 conn(0x7fa1b80fe450 msgr2=0x7fa1b80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:46.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 shutdown_connections 2024-04-25T16:29:46.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:46.492+0000 7fa1bf020700 1 -- 172.21.15.112:0/2749978517 wait complete. 2024-04-25T16:29:47.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:47.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:47.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:47.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:47.073 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:47.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T16:29:47.337 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:48.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:48 smithi112 ceph-mon[29122]: Added host smithi133 2024-04-25T16:29:48.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:48 smithi112 ceph-mon[29122]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:48.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:48 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:48.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:48 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 -- 172.21.15.112:0/4175356818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 msgr2=0x7fb0d8102700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 --2- 172.21.15.112:0/4175356818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8102700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800aa20 tx=0x7fb0c800ad30 comp rx=0 tx=0).stop 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 -- 172.21.15.112:0/4175356818 shutdown_connections 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 --2- 172.21.15.112:0/4175356818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8102700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 -- 172.21.15.112:0/4175356818 >> 172.21.15.112:0/4175356818 conn(0x7fb0d8077c40 msgr2=0x7fb0d8078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 -- 172.21.15.112:0/4175356818 shutdown_connections 2024-04-25T16:29:48.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.875+0000 7fb0df620700 1 -- 172.21.15.112:0/4175356818 wait complete. 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0df620700 1 Processor -- start 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0df620700 1 -- start start 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0df620700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0df620700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d810bd40 con 0x7fb0d81022f0 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0dd3bc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0dd3bc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52912/0 (socket says 172.21.15.112:52912) 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0dd3bc700 1 -- 172.21.15.112:0/1549833414 learned_addr learned my addr 172.21.15.112:0/1549833414 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.876+0000 7fb0dd3bc700 1 -- 172.21.15.112:0/1549833414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d8199c80 con 0x7fb0d81022f0 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.877+0000 7fb0dd3bc700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8199740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8006010 tx=0x7fb0c8005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:48.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.877+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c8005400 con 0x7fb0d81022f0 2024-04-25T16:29:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.877+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c80055a0 con 0x7fb0d81022f0 2024-04-25T16:29:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.877+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb0c800e700 con 0x7fb0d81022f0 2024-04-25T16:29:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.877+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d8199f10 con 0x7fb0d81022f0 2024-04-25T16:29:48.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.877+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d819df70 con 0x7fb0d81022f0 2024-04-25T16:29:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.878+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fb0c800e930 con 0x7fb0d81022f0 2024-04-25T16:29:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.878+0000 7fb0ce7fc700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb0c403d610 0x7fb0c403fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.878+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d81037a0 con 0x7fb0d81022f0 2024-04-25T16:29:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.878+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb0c801e070 con 0x7fb0d81022f0 2024-04-25T16:29:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.878+0000 7fb0dcbbb700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb0c403d610 0x7fb0c403fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:48.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.878+0000 7fb0dcbbb700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb0c403d610 0x7fb0c403fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006fd0 tx=0x7fb0d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:48.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:48.883+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0c800d3e0 con 0x7fb0d81022f0 2024-04-25T16:29:49.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.088+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0d819aaa0 con 0x7fb0c403d610 2024-04-25T16:29:49.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.089+0000 7fb0ce7fc700 1 -- 172.21.15.112:0/1549833414 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fb0d819aaa0 con 0x7fb0c403d610 2024-04-25T16:29:49.089 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:29:49.089 INFO:teuthology.orchestra.run.smithi112.stdout:[{"addr": "172.21.15.112", "hostname": "smithi112", "labels": [], "status": ""}, {"addr": "172.21.15.116", "hostname": "smithi116", "labels": [], "status": ""}, {"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}] 2024-04-25T16:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb0c403d610 msgr2=0x7fb0c403fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb0c403d610 0x7fb0c403fb20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4006fd0 tx=0x7fb0d4006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 msgr2=0x7fb0d8199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8199740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8006010 tx=0x7fb0c8005050 comp rx=0 tx=0).stop 2024-04-25T16:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 shutdown_connections 2024-04-25T16:29:49.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb0c403d610 0x7fb0c403fb20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 --2- 172.21.15.112:0/1549833414 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb0d81022f0 0x7fb0d8199740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 >> 172.21.15.112:0/1549833414 conn(0x7fb0d8077c40 msgr2=0x7fb0d8103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 shutdown_connections 2024-04-25T16:29:49.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:49.091+0000 7fb0df620700 1 -- 172.21.15.112:0/1549833414 wait complete. 2024-04-25T16:29:49.678 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T16:29:49.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T16:29:49.924 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:29:50.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:50 smithi112 ceph-mon[29122]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:50.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:50 smithi112 ceph-mon[29122]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:29:50.313 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:51.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 -- 172.21.15.112:0/94550913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 msgr2=0x7fd6d4102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:51.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 --2- 172.21.15.112:0/94550913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d4102630 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009910 tx=0x7fd6c4009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:51.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 -- 172.21.15.112:0/94550913 shutdown_connections 2024-04-25T16:29:51.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 --2- 172.21.15.112:0/94550913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d4102630 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:51.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 -- 172.21.15.112:0/94550913 >> 172.21.15.112:0/94550913 conn(0x7fd6d4077c20 msgr2=0x7fd6d4078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:51.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 -- 172.21.15.112:0/94550913 shutdown_connections 2024-04-25T16:29:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.389+0000 7fd6d9228700 1 -- 172.21.15.112:0/94550913 wait complete. 2024-04-25T16:29:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.390+0000 7fd6d9228700 1 Processor -- start 2024-04-25T16:29:51.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.390+0000 7fd6d9228700 1 -- start start 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d9228700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d419f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d9228700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d410dfd0 con 0x7fd6d4102220 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d419f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d2d9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d419f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52928/0 (socket says 172.21.15.112:52928) 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d2d9d700 1 -- 172.21.15.112:0/107314423 learned_addr learned my addr 172.21.15.112:0/107314423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d2d9d700 1 -- 172.21.15.112:0/107314423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d419fa50 con 0x7fd6d4102220 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d2d9d700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d419f510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40056d0 tx=0x7fd6c40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:51.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6c4027070 con 0x7fd6d4102220 2024-04-25T16:29:51.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d419fd40 con 0x7fd6d4102220 2024-04-25T16:29:51.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6c400be10 con 0x7fd6d4102220 2024-04-25T16:29:51.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.391+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6c4022d20 con 0x7fd6d4102220 2024-04-25T16:29:51.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.392+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d4199870 con 0x7fd6d4102220 2024-04-25T16:29:51.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.392+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fd6c402a070 con 0x7fd6d4102220 2024-04-25T16:29:51.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.393+0000 7fd6bbfff700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd6bc03d610 0x7fd6bc03fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:51.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.393+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd6c401e030 con 0x7fd6d4102220 2024-04-25T16:29:51.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.393+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d41052b0 con 0x7fd6d4102220 2024-04-25T16:29:51.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.393+0000 7fd6d259c700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd6bc03d610 0x7fd6bc03fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:51.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.393+0000 7fd6d259c700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd6bc03d610 0x7fd6bc03fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:51.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.398+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6c4013070 con 0x7fd6d4102220 2024-04-25T16:29:51.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:51.600+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd6d419ff10 con 0x7fd6d4102220 2024-04-25T16:29:52.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.110+0000 7fd6bbfff700 1 -- 172.21.15.112:0/107314423 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd6c400d340 con 0x7fd6d4102220 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd6bc03d610 msgr2=0x7fd6bc03fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd6bc03d610 0x7fd6bc03fb20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8006fd0 tx=0x7fd6c8006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 msgr2=0x7fd6d419f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d419f510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40056d0 tx=0x7fd6c40057b0 comp rx=0 tx=0).stop 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 shutdown_connections 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd6bc03d610 0x7fd6bc03fb20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 --2- 172.21.15.112:0/107314423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd6d4102220 0x7fd6d419f510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 >> 172.21.15.112:0/107314423 conn(0x7fd6d4077c20 msgr2=0x7fd6d41056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 shutdown_connections 2024-04-25T16:29:52.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:29:52.113+0000 7fd6d9228700 1 -- 172.21.15.112:0/107314423 wait complete. 2024-04-25T16:29:52.115 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-04-25T16:29:52.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:52.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/107314423' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T16:29:52.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:52.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:52.439 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:52.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:52.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:29:52.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:29:52.440 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:29:52.766 INFO:tasks.cephadm:Adding mon.a on smithi112 2024-04-25T16:29:52.766 INFO:tasks.cephadm:Adding mon.b on smithi116 2024-04-25T16:29:52.766 INFO:tasks.cephadm:Adding mon.c on smithi133 2024-04-25T16:29:52.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c' 2024-04-25T16:29:52.996 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:29:53.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:53 smithi112 ceph-mon[29122]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:29:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:53 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/107314423' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T16:29:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:53 smithi112 ceph-mon[29122]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:29:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:53.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.293+0000 7f2c9407e700 1 -- 172.21.15.133:0/1797734511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 msgr2=0x7f2c8c108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.293+0000 7f2c9407e700 1 --2- 172.21.15.133:0/1797734511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c108d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c88009910 tx=0x7f2c88009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.293+0000 7f2c9407e700 1 -- 172.21.15.133:0/1797734511 shutdown_connections 2024-04-25T16:29:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.293+0000 7f2c9407e700 1 --2- 172.21.15.133:0/1797734511 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c108d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:54.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.293+0000 7f2c9407e700 1 -- 172.21.15.133:0/1797734511 >> 172.21.15.133:0/1797734511 conn(0x7f2c8c078460 msgr2=0x7f2c8c0788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.294+0000 7f2c9407e700 1 -- 172.21.15.133:0/1797734511 shutdown_connections 2024-04-25T16:29:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.294+0000 7f2c9407e700 1 -- 172.21.15.133:0/1797734511 wait complete. 2024-04-25T16:29:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.294+0000 7f2c9407e700 1 Processor -- start 2024-04-25T16:29:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.294+0000 7f2c9407e700 1 -- start start 2024-04-25T16:29:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.294+0000 7f2c9407e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:54.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c9407e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c10ce80 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c91e1a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c91e1a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.133:35138/0 (socket says 172.21.15.133:35138) 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c91e1a700 1 -- 172.21.15.133:0/2300396667 learned_addr learned my addr 172.21.15.133:0/2300396667 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c91e1a700 1 -- 172.21.15.133:0/2300396667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c198c30 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c91e1a700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c1986f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c880092e0 tx=0x7f2c88005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c88026070 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c198f20 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.295+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c8800b6f0 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.296+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c880037f0 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.296+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c19cf20 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.296+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c8c104040 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.297+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f2c88029070 con 0x7f2c8c108930 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.297+0000 7f2c82ffd700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2c7803d610 0x7f2c7803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:54.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.297+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2c8801e070 con 0x7f2c8c108930 2024-04-25T16:29:54.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.297+0000 7f2c91619700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2c7803d610 0x7f2c7803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:54.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.298+0000 7f2c91619700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2c7803d610 0x7f2c7803fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c006fd0 tx=0x7f2c7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:54.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.303+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c88013070 con 0x7f2c8c108930 2024-04-25T16:29:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.512+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}) -- 0x7f2c8c199a50 con 0x7f2c7803d610 2024-04-25T16:29:54.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.520+0000 7f2c82ffd700 1 -- 172.21.15.133:0/2300396667 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2c8c199a50 con 0x7f2c7803d610 2024-04-25T16:29:54.519 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mon update... 2024-04-25T16:29:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2c7803d610 msgr2=0x7f2c7803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2c7803d610 0x7f2c7803fb20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c006fd0 tx=0x7f2c7c006e40 comp rx=0 tx=0).stop 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 msgr2=0x7f2c8c1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c1986f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2c880092e0 tx=0x7f2c88005780 comp rx=0 tx=0).stop 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 shutdown_connections 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2c7803d610 0x7f2c7803fb20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 --2- 172.21.15.133:0/2300396667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2c8c108930 0x7f2c8c1986f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 >> 172.21.15.133:0/2300396667 conn(0x7f2c8c078460 msgr2=0x7f2c8c0ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 shutdown_connections 2024-04-25T16:29:54.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:54.522+0000 7f2c9407e700 1 -- 172.21.15.133:0/2300396667 wait complete. 2024-04-25T16:29:54.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:54 smithi112 ceph-mon[29122]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:29:54.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:54 smithi112 ceph-mon[29122]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:29:54.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:54 smithi112 ceph-mon[29122]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:29:54.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:54 smithi112 ceph-mon[29122]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:55.138 DEBUG:teuthology.orchestra.run.smithi116:mon.b> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T16:29:55.141 DEBUG:teuthology.orchestra.run.smithi133:mon.c> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T16:29:55.144 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T16:29:55.144 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T16:29:55.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c;count:3 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: Deploying daemon mon.c on smithi133 2024-04-25T16:29:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:55 smithi112 ceph-mon[29122]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:57.162 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:29:58.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:29:58 smithi112 ceph-mon[29122]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:29:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 -- 172.21.15.133:0/54514858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 msgr2=0x7f1ab0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 --2- 172.21.15.133:0/54514858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab0108f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009910 tx=0x7f1aa0009c20 comp rx=0 tx=0).stop 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 -- 172.21.15.133:0/54514858 shutdown_connections 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 --2- 172.21.15.133:0/54514858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab0108f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 -- 172.21.15.133:0/54514858 >> 172.21.15.133:0/54514858 conn(0x7f1ab00fddf0 msgr2=0x7f1ab0100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 -- 172.21.15.133:0/54514858 shutdown_connections 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.229+0000 7f1ab4e44700 1 -- 172.21.15.133:0/54514858 wait complete. 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.230+0000 7f1ab4e44700 1 Processor -- start 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.230+0000 7f1ab4e44700 1 -- start start 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.230+0000 7f1ab4e44700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab01030e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.230+0000 7f1ab4e44700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab010e190 con 0x7f1ab0108b10 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1aae59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab01030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1aae59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab01030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.133:52250/0 (socket says 172.21.15.133:52250) 2024-04-25T16:29:59.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1aae59c700 1 -- 172.21.15.133:0/569151836 learned_addr learned my addr 172.21.15.133:0/569151836 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1aae59c700 1 -- 172.21.15.133:0/569151836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab0104f30 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1aae59c700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab01030e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000b5c0 tx=0x7f1aa0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa0026070 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab0103620 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1aa000b750 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1aa00037d0 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.231+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab0103a20 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.232+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab0064cf0 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.232+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f1aa0029070 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.233+0000 7f1a9f7fe700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1a9803d610 0x7f1a9803fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.233+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1aa001e070 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.233+0000 7f1aadd9b700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1a9803d610 0x7f1a9803fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.233+0000 7f1aadd9b700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1a9803d610 0x7f1a9803fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4006fd0 tx=0x7f1aa4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.238+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1aa0013070 con 0x7f1ab0108b10 2024-04-25T16:29:59.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.290+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1aa00142b0 con 0x7f1ab0108b10 2024-04-25T16:29:59.445 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:29:59 smithi133 systemd[1]: Started Ceph mon.c for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:29:59.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:29:59.547+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1ab0105260 con 0x7f1ab0108b10 2024-04-25T16:30:04.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.302+0000 7f1a9f7fe700 1 -- 172.21.15.133:0/569151836 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f1aa0030090 con 0x7f1ab0108b10 2024-04-25T16:30:04.302 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:30:04.302 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","modified":"2024-04-25T16:29:59.290442Z","created":"2024-04-25T16:28:09.103195Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1a9803d610 msgr2=0x7f1a9803fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1a9803d610 0x7f1a9803fb20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4006fd0 tx=0x7f1aa4006e40 comp rx=0 tx=0).stop 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 msgr2=0x7f1ab01030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab01030e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000b5c0 tx=0x7f1aa0005780 comp rx=0 tx=0).stop 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 shutdown_connections 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1a9803d610 0x7f1a9803fb20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 --2- 172.21.15.133:0/569151836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ab0108b10 0x7f1ab01030e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 >> 172.21.15.133:0/569151836 conn(0x7f1ab00fddf0 msgr2=0x7f1ab0105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:04.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 shutdown_connections 2024-04-25T16:30:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:04.305+0000 7f1ab4e44700 1 -- 172.21.15.133:0/569151836 wait complete. 2024-04-25T16:30:04.306 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2024-04-25T16:30:04.595 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: mon.a calling monitor election 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/569151836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: mon.c calling monitor election 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.596 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: monmap epoch 2 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: last_changed 2024-04-25T16:29:59.290442+0000 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: min_mon_release 19 (squid) 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: election_strategy: 1 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: fsmap 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: mgrmap e13: x(active, since 49s) 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: overall HEALTH_OK 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:04.597 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:04.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:30:04.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: mon.a calling monitor election 2024-04-25T16:30:04.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/569151836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:04.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: mon.c calling monitor election 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: monmap epoch 2 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: last_changed 2024-04-25T16:29:59.290442+0000 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: min_mon_release 19 (squid) 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: election_strategy: 1 2024-04-25T16:30:04.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: fsmap 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: mgrmap e13: x(active, since 49s) 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: overall HEALTH_OK 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:04.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:05.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:05.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:05.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:05.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:05 smithi133 ceph-mon[32691]: Deploying daemon mon.b on smithi116 2024-04-25T16:30:05.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:05 smithi133 ceph-mon[32691]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:05.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:05.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:05.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:05.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:05.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:05 smithi112 ceph-mon[29122]: Deploying daemon mon.b on smithi116 2024-04-25T16:30:05.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:05 smithi112 ceph-mon[29122]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:05.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:05.901 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T16:30:05.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T16:30:06.141 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:30:07.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.617+0000 7fc460585700 1 -- 172.21.15.133:0/840992019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 msgr2=0x7fc458108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.617+0000 7fc460585700 1 --2- 172.21.15.133:0/840992019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc458108f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc45000b0a0 tx=0x7fc45000b3b0 comp rx=0 tx=0).stop 2024-04-25T16:30:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.617+0000 7fc460585700 1 -- 172.21.15.133:0/840992019 shutdown_connections 2024-04-25T16:30:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.617+0000 7fc460585700 1 --2- 172.21.15.133:0/840992019 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc458108f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.617+0000 7fc460585700 1 -- 172.21.15.133:0/840992019 >> 172.21.15.133:0/840992019 conn(0x7fc458077ce0 msgr2=0x7fc458078120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.617+0000 7fc460585700 1 -- 172.21.15.133:0/840992019 shutdown_connections 2024-04-25T16:30:07.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.618+0000 7fc460585700 1 -- 172.21.15.133:0/840992019 wait complete. 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.618+0000 7fc460585700 1 Processor -- start 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.618+0000 7fc460585700 1 -- start start 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc460585700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc45819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc460585700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc460585700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc45810e1e0 con 0x7fc458108b60 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc460585700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc45810e060 con 0x7fc45819fcb0 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45e321700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc45819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44418/0 (socket says 172.21.15.133:44418) 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 -- 172.21.15.133:0/1984045721 learned_addr learned my addr 172.21.15.133:0/1984045721 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 -- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 msgr2=0x7fc458199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T16:30:07.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 -- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 msgr2=0x7fc458199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45db20700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45e321700 1 -- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 msgr2=0x7fc458199990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45e321700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.619+0000 7fc45e321700 1 -- 172.21.15.133:0/1984045721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4581a0280 con 0x7fc458108b60 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.620+0000 7fc45e321700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc45819f770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc45000d010 tx=0x7fc450007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.620+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc450017040 con 0x7fc458108b60 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.620+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4500128e0 con 0x7fc458108b60 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.620+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc458199f30 con 0x7fc458108b60 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.620+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc45001d8c0 con 0x7fc458108b60 2024-04-25T16:30:07.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.620+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45819a490 con 0x7fc458108b60 2024-04-25T16:30:07.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.621+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc458105270 con 0x7fc458108b60 2024-04-25T16:30:07.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.621+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fc450024020 con 0x7fc458108b60 2024-04-25T16:30:07.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.621+0000 7fc44f7fe700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc44403d3b0 0x7fc44403f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:07.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.621+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc45000f050 con 0x7fc458108b60 2024-04-25T16:30:07.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.621+0000 7fc45db20700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc44403d3b0 0x7fc44403f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:07.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.622+0000 7fc45db20700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc44403d3b0 0x7fc44403f8c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc4581000e0 tx=0x7fc448008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:07.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.627+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc45002b020 con 0x7fc458108b60 2024-04-25T16:30:07.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.906+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc45819aec0 con 0x7fc458108b60 2024-04-25T16:30:07.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.907+0000 7fc44f7fe700 1 -- 172.21.15.133:0/1984045721 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fc45001c7e0 con 0x7fc458108b60 2024-04-25T16:30:07.907 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:30:07.908 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","modified":"2024-04-25T16:29:59.290442Z","created":"2024-04-25T16:28:09.103195Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T16:30:07.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc44403d3b0 msgr2=0x7fc44403f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc44403d3b0 0x7fc44403f8c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc4581000e0 tx=0x7fc448008040 comp rx=0 tx=0).stop 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 msgr2=0x7fc45819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc45819f770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc45000d010 tx=0x7fc450007a50 comp rx=0 tx=0).stop 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 shutdown_connections 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc44403d3b0 0x7fc44403f8c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc458108b60 0x7fc45819f770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 --2- 172.21.15.133:0/1984045721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc45819fcb0 0x7fc458199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 >> 172.21.15.133:0/1984045721 conn(0x7fc458077ce0 msgr2=0x7fc4580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 shutdown_connections 2024-04-25T16:30:07.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:07.909+0000 7fc460585700 1 -- 172.21.15.133:0/1984045721 wait complete. 2024-04-25T16:30:07.911 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 2 2024-04-25T16:30:08.104 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 podman[32692]: 2024-04-25 16:30:08.102166698 +0000 UTC m=+0.111071581 container create dd884f0c9137ae8dff8c5059cfe190126d52291a69fb06d309f8198626570551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T16:30:08.105 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 podman[32692]: 2024-04-25 16:30:08.008819372 +0000 UTC m=+0.017724280 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:30:08.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:08 smithi133 ceph-mon[32691]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:08 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/1984045721' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:08 smithi112 ceph-mon[29122]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:08 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/1984045721' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:08.368 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 podman[32692]: 2024-04-25 16:30:08.306966148 +0000 UTC m=+0.315871047 container init dd884f0c9137ae8dff8c5059cfe190126d52291a69fb06d309f8198626570551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:30:08.368 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 podman[32692]: 2024-04-25 16:30:08.310209014 +0000 UTC m=+0.319113900 container start dd884f0c9137ae8dff8c5059cfe190126d52291a69fb06d309f8198626570551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:30:08.368 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T16:30:08.368 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T16:30:08.368 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pidfile_write: ignore empty --pid-file 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: load: jerasure load: lrc 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: RocksDB version: 7.9.2 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Git sha 0 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: DB SUMMARY 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: DB Session ID: ZN4QWWKJ1AWU9OMBM30C 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: CURRENT file: CURRENT 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.error_if_exists: 0 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.create_if_missing: 0 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.paranoid_checks: 1 2024-04-25T16:30:08.369 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T16:30:08.370 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T16:30:08.370 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T16:30:08.370 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.env: 0x5635ffa4d500 2024-04-25T16:30:08.370 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T16:30:08.370 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.info_log: 0x563600bf9fc0 2024-04-25T16:30:08.370 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.statistics: (nil) 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.use_fsync: 0 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_log_file_size: 0 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.allow_fallocate: 1 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T16:30:08.371 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.use_direct_reads: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.db_log_dir: 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.wal_dir: 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T16:30:08.372 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.write_buffer_manager: 0x563600bca8c0 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.unordered_write: 0 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.row_cache: None 2024-04-25T16:30:08.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.wal_filter: None 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.two_write_queues: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.wal_compression: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.atomic_flush: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.log_readahead_size: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T16:30:08.374 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_background_jobs: 2 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_background_compactions: -1 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_subcompactions: 1 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T16:30:08.375 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_open_files: -1 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_background_flushes: -1 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Compression algorithms supported: 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kZSTD supported: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kXpressCompression supported: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kZlibCompression supported: 1 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kSnappyCompression supported: 1 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kLZ4Compression supported: 1 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: kBZip2Compression supported: 0 2024-04-25T16:30:08.376 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.merge_operator: 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_filter: None 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563600d001a0) 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout: cache_index_and_filter_blocks: 1 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T16:30:08.377 INFO:journalctl@ceph.mon.b.smithi116.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: pin_top_level_index_and_filter: 1 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: index_type: 0 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: data_block_index_type: 0 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: index_shortening: 1 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: checksum: 4 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: no_block_cache: 0 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_cache: 0x563600bfd1f0 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_cache_name: BinnedLRUCache 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_cache_options: 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: capacity : 536870912 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: num_shard_bits : 4 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: strict_capacity_limit : 0 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: high_pri_pool_ratio: 0.000 2024-04-25T16:30:08.378 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_cache_compressed: (nil) 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: persistent_cache: (nil) 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_size: 4096 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_size_deviation: 10 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_restart_interval: 16 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: index_block_restart_interval: 1 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: metadata_block_size: 4096 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: partition_filters: 0 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: use_delta_encoding: 1 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: filter_policy: bloomfilter 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: whole_key_filtering: 1 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: verify_compression: 0 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: read_amp_bytes_per_bit: 0 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: format_version: 5 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: enable_index_compression: 1 2024-04-25T16:30:08.379 INFO:journalctl@ceph.mon.b.smithi116.stdout: block_align: 0 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout: max_auto_readahead_size: 262144 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout: prepopulate_block_cache: 0 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout: initial_auto_readahead_size: 8192 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression: NoCompression 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.num_levels: 7 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T16:30:08.380 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T16:30:08.381 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T16:30:08.382 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T16:30:08.383 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T16:30:08.383 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T16:30:08.384 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.inplace_update_support: 0 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.bloom_locality: 0 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.max_successive_merges: 0 2024-04-25T16:30:08.385 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.ttl: 2592000 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.enable_blob_files: false 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.min_blob_size: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T16:30:08.386 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5794c852-3c5a-4639-8cfa-8edeb18320ed 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714062608350351, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714062608350776, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714062608, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5794c852-3c5a-4639-8cfa-8edeb18320ed", "db_session_id": "ZN4QWWKJ1AWU9OMBM30C", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714062608350855, "job": 1, "event": "recovery_finished"} 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563600be1c00 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: DB pointer 0x5636059d4000 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T16:30:08.387 INFO:journalctl@ceph.mon.b.smithi116.stdout: ** DB Stats ** 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: ** Compaction Stats [default] ** 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: 2024-04-25T16:30:08.388 INFO:journalctl@ceph.mon.b.smithi116.stdout: ** Compaction Stats [default] ** 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Cumulative compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Interval compaction: 0.00 GB write, 0.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T16:30:08.389 INFO:journalctl@ceph.mon.b.smithi116.stdout: Block cache BinnedLRUCache@0x563600bfd1f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: using public_addr v2:172.21.15.116:0/0 -> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: starting mon.b rank -1 at public addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] at bind addrs [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(???) e0 preinit fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: e1 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: legacy client fscid: -1 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: 2024-04-25T16:30:08.390 INFO:journalctl@ceph.mon.b.smithi116.stdout: No filesystems configured 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.391 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi116", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Deploying cephadm binary to smithi133 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.392 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Added host smithi133 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/107314423' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:30:08.393 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/107314423' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Saving service mon spec with placement smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c;count:3 2024-04-25T16:30:08.394 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Deploying daemon mon.c on smithi133 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.a calling monitor election 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/569151836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.395 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.c calling monitor election 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: monmap epoch 2 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: last_changed 2024-04-25T16:29:59.290442+0000 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: min_mon_release 19 (squid) 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: election_strategy: 1 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: fsmap 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:08.396 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mgrmap e13: x(active, since 49s) 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: overall HEALTH_OK 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: Deploying daemon mon.b on smithi116 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/1984045721' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 ceph-mon[32712]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T16:30:08.397 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 bash[32692]: dd884f0c9137ae8dff8c5059cfe190126d52291a69fb06d309f8198626570551 2024-04-25T16:30:08.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:08 smithi116 systemd[1]: Started Ceph mon.b for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:30:09.400 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T16:30:09.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T16:30:09.641 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: mon.a calling monitor election 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: mon.c calling monitor election 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: mon.b calling monitor election 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: monmap epoch 3 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: last_changed 2024-04-25T16:30:08.375299+0000 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: min_mon_release 19 (squid) 2024-04-25T16:30:13.598 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: election_strategy: 1 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: 2: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.b 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: fsmap 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: mgrmap e13: x(active, since 58s) 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: overall HEALTH_OK 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:13.599 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: mon.a calling monitor election 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: mon.c calling monitor election 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: mon.b calling monitor election 2024-04-25T16:30:13.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: monmap epoch 3 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: last_changed 2024-04-25T16:30:08.375299+0000 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: min_mon_release 19 (squid) 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: election_strategy: 1 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: 2: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.b 2024-04-25T16:30:13.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: fsmap 2024-04-25T16:30:13.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:13.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: mgrmap e13: x(active, since 58s) 2024-04-25T16:30:13.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: overall HEALTH_OK 2024-04-25T16:30:13.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:13.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: mon.a calling monitor election 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: mon.c calling monitor election 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: mon.b calling monitor election 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: monmap epoch 3 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: last_changed 2024-04-25T16:30:08.375299+0000 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: created 2024-04-25T16:28:09.103195+0000 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: min_mon_release 19 (squid) 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: election_strategy: 1 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: 0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: 1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.c 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: 2: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.b 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: fsmap 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: mgrmap e13: x(active, since 58s) 2024-04-25T16:30:13.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: overall HEALTH_OK 2024-04-25T16:30:13.762 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:13.762 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.083+0000 7f478baf5700 1 -- 172.21.15.133:0/1781295581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 msgr2=0x7f4770005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.083+0000 7f478baf5700 1 --2- 172.21.15.133:0/1781295581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f4770005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4784102940 tx=0x7f478000feb0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 -- 172.21.15.133:0/1781295581 shutdown_connections 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 --2- 172.21.15.133:0/1781295581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f4770005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 -- 172.21.15.133:0/1781295581 >> 172.21.15.133:0/1781295581 conn(0x7f47840796b0 msgr2=0x7f4784079af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 -- 172.21.15.133:0/1781295581 shutdown_connections 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 -- 172.21.15.133:0/1781295581 wait complete. 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 Processor -- start 2024-04-25T16:30:14.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.084+0000 7f478baf5700 1 -- start start 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478baf5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f478419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478baf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 0x7f4784199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478baf5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478baf5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f478410a9f0 con 0x7f47700035c0 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478baf5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f478410a870 con 0x7f478419fca0 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478baf5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f478410ab70 con 0x7f4784199ec0 2024-04-25T16:30:14.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44426/0 (socket says 172.21.15.133:44426) 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 -- 172.21.15.133:0/2302132709 learned_addr learned my addr 172.21.15.133:0/2302132709 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789891700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f478419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789090700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 0x7f4784199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 msgr2=0x7f478419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 msgr2=0x7f478419a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f478a092700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789090700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 msgr2=0x7f4784199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789090700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 msgr2=0x7f4784199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789090700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 0x7f4784199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789090700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 0x7f4784199980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789891700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 msgr2=0x7f478419a2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789891700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789891700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 msgr2=0x7f4784199980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789891700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 0x7f4784199980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.085+0000 7f4789891700 1 -- 172.21.15.133:0/2302132709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47841a65f0 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.086+0000 7f4789891700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f478419f760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4780000c00 tx=0x7f47800053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.086+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780005020 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.086+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47800051c0 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.086+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4780003bc0 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.086+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47841a6820 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.086+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841a6d80 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.087+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f4780023070 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.087+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784064cf0 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.087+0000 7f477affd700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f476803daa0 0x7f476803ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.087+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f47800537d0 con 0x7f47700035c0 2024-04-25T16:30:14.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.088+0000 7f4789090700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f476803daa0 0x7f476803ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:14.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.088+0000 7f4789090700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f476803daa0 0x7f476803ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47840fff90 tx=0x7f4774008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:14.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.093+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4780028020 con 0x7f47700035c0 2024-04-25T16:30:14.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.407+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f47841a7080 con 0x7f47700035c0 2024-04-25T16:30:14.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.408+0000 7f477affd700 1 -- 172.21.15.133:0/2302132709 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f478001a390 con 0x7f47700035c0 2024-04-25T16:30:14.408 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-04-25T16:30:14.408 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","modified":"2024-04-25T16:30:08.375299Z","created":"2024-04-25T16:28:09.103195Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:3300","nonce":0},{"type":"v1","addr":"172.21.15.116:6789","nonce":0}]},"addr":"172.21.15.116:6789/0","public_addr":"172.21.15.116:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T16:30:14.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.410+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f476803daa0 msgr2=0x7f476803ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:14.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.410+0000 7f478baf5700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f476803daa0 0x7f476803ffb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f47840fff90 tx=0x7f4774008040 comp rx=0 tx=0).stop 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.410+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 msgr2=0x7f478419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.410+0000 7f478baf5700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f478419f760 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4780000c00 tx=0x7f47800053b0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 shutdown_connections 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4784199ec0 0x7f478419a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47700035c0 0x7f478419f760 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f476803daa0 0x7f476803ffb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 --2- 172.21.15.133:0/2302132709 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f478419fca0 0x7f4784199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 >> 172.21.15.133:0/2302132709 conn(0x7f47840796b0 msgr2=0x7f4784101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 shutdown_connections 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:14.411+0000 7f478baf5700 1 -- 172.21.15.133:0/2302132709 wait complete. 2024-04-25T16:30:14.411 INFO:teuthology.orchestra.run.smithi133.stderr:dumped monmap epoch 3 2024-04-25T16:30:14.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:14 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:14 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:14 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:14.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:14 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:14.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:14 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/2302132709' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:14.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:14 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:14 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:14 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:14.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:14 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:14.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:14 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/2302132709' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:14.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:14 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:14 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:14.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:14 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:14.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:14 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:30:14.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:14 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/2302132709' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T16:30:15.441 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T16:30:15.441 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T16:30:15.684 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:30:16.078 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:16.079 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:16.080 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:16.202 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:16.261 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:17.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:16 smithi112 ceph-mon[29122]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:16 smithi112 ceph-mon[29122]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:16 smithi112 ceph-mon[29122]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:16 smithi112 ceph-mon[29122]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T16:30:17.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:16 smithi112 ceph-mon[29122]: Reconfiguring daemon mon.a on smithi112 2024-04-25T16:30:17.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:16 smithi133 ceph-mon[32691]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:16 smithi133 ceph-mon[32691]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:16 smithi133 ceph-mon[32691]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:16 smithi133 ceph-mon[32691]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T16:30:17.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:16 smithi133 ceph-mon[32691]: Reconfiguring daemon mon.a on smithi112 2024-04-25T16:30:17.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:16 smithi116 ceph-mon[32712]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:16 smithi116 ceph-mon[32712]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:16 smithi116 ceph-mon[32712]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:30:17.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:16 smithi116 ceph-mon[32712]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T16:30:17.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:16 smithi116 ceph-mon[32712]: Reconfiguring daemon mon.a on smithi112 2024-04-25T16:30:18.047 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:17 smithi112 ceph-mon[29122]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:18.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:18.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:18.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:18.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:18.048 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:17 smithi133 ceph-mon[32691]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:18.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:18.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:18.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:18.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:18.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:18.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:17 smithi116 ceph-mon[32712]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:18.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:18.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:18.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:18.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:18.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 -- 172.21.15.112:0/1243268169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 msgr2=0x7fdf08111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/1243268169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf08111040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000b0a0 tx=0x7fdf0000b3b0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 -- 172.21.15.112:0/1243268169 shutdown_connections 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/1243268169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf08111040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/1243268169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 0x7fdf081095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/1243268169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf08102660 0x7fdf08102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 -- 172.21.15.112:0/1243268169 >> 172.21.15.112:0/1243268169 conn(0x7fdf080fe3f0 msgr2=0x7fdf08100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 -- 172.21.15.112:0/1243268169 shutdown_connections 2024-04-25T16:30:18.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.286+0000 7fdf0f52a700 1 -- 172.21.15.112:0/1243268169 wait complete. 2024-04-25T16:30:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 Processor -- start 2024-04-25T16:30:18.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 -- start start 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf08102660 0x7fdf0819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf0819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 0x7fdf0819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf081144f0 con 0x7fdf08102fb0 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08114370 con 0x7fdf08102660 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0f52a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08114670 con 0x7fdf081091d0 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.287+0000 7fdf0cac5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf0819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf0819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58790/0 (socket says 172.21.15.112:58790) 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 -- 172.21.15.112:0/4033923682 learned_addr learned my addr 172.21.15.112:0/4033923682 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0d2c6700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf08102660 0x7fdf0819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0dac7700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 0x7fdf0819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 -- 172.21.15.112:0/4033923682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 msgr2=0x7fdf0819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:18.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 0x7fdf0819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 -- 172.21.15.112:0/4033923682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf08102660 msgr2=0x7fdf0819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf08102660 0x7fdf0819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 -- 172.21.15.112:0/4033923682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0819fc50 con 0x7fdf08102fb0 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0dac7700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 0x7fdf0819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0cac5700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf0819b320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d980 tx=0x7fdef800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8017070 con 0x7fdf08102fb0 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdef8009de0 con 0x7fdf08102fb0 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0819ff40 con 0x7fdf08102fb0 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef800f3c0 con 0x7fdf08102fb0 2024-04-25T16:30:18.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.288+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf08103770 con 0x7fdf08102fb0 2024-04-25T16:30:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.289+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fdef800f560 con 0x7fdf08102fb0 2024-04-25T16:30:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.289+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0810c770 con 0x7fdf08102fb0 2024-04-25T16:30:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.290+0000 7fdefe7fc700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdef403d850 0x7fdef403fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.290+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdef8051630 con 0x7fdf08102fb0 2024-04-25T16:30:18.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.290+0000 7fdf0d2c6700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdef403d850 0x7fdef403fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:18.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.290+0000 7fdf0d2c6700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdef403d850 0x7fdef403fd60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf0806c940 tx=0x7fdf04009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:18.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.295+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdef800f9a0 con 0x7fdf08102fb0 2024-04-25T16:30:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.499+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdf0819c140 con 0x7fdf08102fb0 2024-04-25T16:30:18.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.500+0000 7fdefe7fc700 1 -- 172.21.15.112:0/4033923682 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fdef8010460 con 0x7fdf08102fb0 2024-04-25T16:30:18.500 INFO:teuthology.orchestra.run.smithi112.stdout:# minimal ceph.conf for a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:18.500 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T16:30:18.500 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:30:18.500 INFO:teuthology.orchestra.run.smithi112.stdout: mon_host = [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdef403d850 msgr2=0x7fdef403fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdef403d850 0x7fdef403fd60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf0806c940 tx=0x7fdf04009190 comp rx=0 tx=0).stop 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 msgr2=0x7fdf0819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf0819b320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d980 tx=0x7fdef800dc90 comp rx=0 tx=0).stop 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 shutdown_connections 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdf081091d0 0x7fdf0819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdef403d850 0x7fdef403fd60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf08102660 0x7fdf0819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 --2- 172.21.15.112:0/4033923682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf08102fb0 0x7fdf0819b320 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 >> 172.21.15.112:0/4033923682 conn(0x7fdf080fe3f0 msgr2=0x7fdf08106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 shutdown_connections 2024-04-25T16:30:18.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:18.502+0000 7fdf0f52a700 1 -- 172.21.15.112:0/4033923682 wait complete. 2024-04-25T16:30:19.112 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T16:30:19.112 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:30:19.112 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T16:30:19.138 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:30:19.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:30:19.198 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:18 smithi112 ceph-mon[29122]: Reconfiguring mon.b (monmap changed)... 2024-04-25T16:30:19.198 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:18 smithi112 ceph-mon[29122]: Reconfiguring daemon mon.b on smithi116 2024-04-25T16:30:19.198 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:18 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4033923682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:19.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:18 smithi133 ceph-mon[32691]: Reconfiguring mon.b (monmap changed)... 2024-04-25T16:30:19.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:18 smithi133 ceph-mon[32691]: Reconfiguring daemon mon.b on smithi116 2024-04-25T16:30:19.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:18 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4033923682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:19.202 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:30:19.202 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T16:30:19.225 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:18 smithi116 ceph-mon[32712]: Reconfiguring mon.b (monmap changed)... 2024-04-25T16:30:19.225 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:18 smithi116 ceph-mon[32712]: Reconfiguring daemon mon.b on smithi116 2024-04-25T16:30:19.225 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:18 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4033923682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:19.229 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:30:19.229 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:30:19.294 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:30:19.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T16:30:19.325 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:30:19.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:30:19.393 INFO:tasks.cephadm:Adding mgr.x on smithi112 2024-04-25T16:30:19.393 INFO:tasks.cephadm:Adding mgr.y on smithi116 2024-04-25T16:30:19.393 INFO:tasks.cephadm:Adding mgr.z on smithi133 2024-04-25T16:30:19.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi112=x;smithi116=y;smithi133=z' 2024-04-25T16:30:19.669 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:30:20.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:19 smithi133 ceph-mon[32691]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:20.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:20.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:20.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:20.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:20.050 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:20.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:19 smithi116 ceph-mon[32712]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:20.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:20.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:20.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:20.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:20.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:19 smithi112 ceph-mon[29122]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:20.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:20.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:20.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T16:30:20.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T16:30:20.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:21.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:20 smithi133 ceph-mon[32691]: Reconfiguring mon.c (monmap changed)... 2024-04-25T16:30:21.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:20 smithi133 ceph-mon[32691]: Reconfiguring daemon mon.c on smithi133 2024-04-25T16:30:21.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:20 smithi116 ceph-mon[32712]: Reconfiguring mon.c (monmap changed)... 2024-04-25T16:30:21.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:20 smithi116 ceph-mon[32712]: Reconfiguring daemon mon.c on smithi133 2024-04-25T16:30:21.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:20 smithi112 ceph-mon[29122]: Reconfiguring mon.c (monmap changed)... 2024-04-25T16:30:21.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:20 smithi112 ceph-mon[29122]: Reconfiguring daemon mon.c on smithi133 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 -- 172.21.15.133:0/708996700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104bb0 msgr2=0x7fd9f810b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/708996700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104bb0 0x7fd9f810b460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000b0a0 tx=0x7fd9f000b3b0 comp rx=0 tx=0).stop 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 -- 172.21.15.133:0/708996700 shutdown_connections 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/708996700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104bb0 0x7fd9f810b460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/708996700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9f81030e0 0x7fd9f81034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/708996700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f8104210 0x7fd9f8104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 -- 172.21.15.133:0/708996700 >> 172.21.15.133:0/708996700 conn(0x7fd9f80fe740 msgr2=0x7fd9f8100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 -- 172.21.15.133:0/708996700 shutdown_connections 2024-04-25T16:30:21.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.943+0000 7fd9fcbce700 1 -- 172.21.15.133:0/708996700 wait complete. 2024-04-25T16:30:21.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 Processor -- start 2024-04-25T16:30:21.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 -- start start 2024-04-25T16:30:21.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f81030e0 0x7fd9f8076340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:21.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 0x7fd9f819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:21.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9f8104bb0 0x7fd9f81a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810eb80 con 0x7fd9f8104bb0 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810ea00 con 0x7fd9f81030e0 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9fcbce700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f810ed00 con 0x7fd9f8104210 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 0x7fd9f819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 0x7fd9f819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43544/0 (socket says 172.21.15.133:43544) 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 -- 172.21.15.133:0/1859016774 learned_addr learned my addr 172.21.15.133:0/1859016774 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f6d9d700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9f8104bb0 0x7fd9f81a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f659c700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f81030e0 0x7fd9f8076340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 -- 172.21.15.133:0/1859016774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f81030e0 msgr2=0x7fd9f8076340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f81030e0 0x7fd9f8076340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 -- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9f8104bb0 msgr2=0x7fd9f81a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.944+0000 7fd9f5d9b700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9f8104bb0 0x7fd9f81a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9f5d9b700 1 -- 172.21.15.133:0/1859016774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f81a2290 con 0x7fd9f8104210 2024-04-25T16:30:21.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9f659c700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f81030e0 0x7fd9f8076340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9f5d9b700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 0x7fd9f819f4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00d980 tx=0x7fd9ec00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec017070 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ec005020 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f81a2580 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec009b50 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.945+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f81a2a70 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.946+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7fd9ec009cf0 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.946+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f8051190 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.946+0000 7fd9e77fe700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd9e003d850 0x7fd9e003fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.946+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd9ec051660 con 0x7fd9f8104210 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.946+0000 7fd9f659c700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd9e003d850 0x7fd9e003fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:21.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.947+0000 7fd9f659c700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd9e003d850 0x7fd9e003fd60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8104110 tx=0x7fd9e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:21.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:21.952+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd9ec020020 con 0x7fd9f8104210 2024-04-25T16:30:22.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.159+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi116=y;smithi133=z", "target": ["mon-mgr", ""]}) -- 0x7fd9f8063b60 con 0x7fd9e003d850 2024-04-25T16:30:22.159 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:21 smithi133 ceph-mon[32691]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:22.159 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:22.159 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:22.159 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:22.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.171+0000 7fd9e77fe700 1 -- 172.21.15.133:0/1859016774 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd9f8063b60 con 0x7fd9e003d850 2024-04-25T16:30:22.170 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mgr update... 2024-04-25T16:30:22.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd9e003d850 msgr2=0x7fd9e003fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd9e003d850 0x7fd9e003fd60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8104110 tx=0x7fd9e8009260 comp rx=0 tx=0).stop 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 msgr2=0x7fd9f819f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 0x7fd9f819f4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00d980 tx=0x7fd9ec00dc90 comp rx=0 tx=0).stop 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 shutdown_connections 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9f8104bb0 0x7fd9f81a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd9e003d850 0x7fd9e003fd60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9f81030e0 0x7fd9f8076340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 --2- 172.21.15.133:0/1859016774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9f8104210 0x7fd9f819f4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 >> 172.21.15.133:0/1859016774 conn(0x7fd9f80fe740 msgr2=0x7fd9f8107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 shutdown_connections 2024-04-25T16:30:22.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:30:22.173+0000 7fd9fcbce700 1 -- 172.21.15.133:0/1859016774 wait complete. 2024-04-25T16:30:22.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:21 smithi116 ceph-mon[32712]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:22.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:22.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:22.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:21 smithi112 ceph-mon[29122]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:22.733 DEBUG:teuthology.orchestra.run.smithi116:mgr.y> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T16:30:22.736 DEBUG:teuthology.orchestra.run.smithi133:mgr.z> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T16:30:22.738 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T16:30:22.738 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:30:22.738 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:30:22.757 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:30:22.757 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 16:29:27.170667372 +0000 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 16:24:38.398796009 +0000 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 16:24:38.398796009 +0000 2024-04-25T16:30:22.811 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 16:24:38.398796009 +0000 2024-04-25T16:30:22.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:30:22.874 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T16:30:22.874 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T16:30:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000279819 s, 1.8 MB/s 2024-04-25T16:30:22.876 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:30:22.934 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T16:30:22.989 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:30:22.989 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:22.989 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T16:30:22.989 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:22.989 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:22.989 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 16:29:27.203666482 +0000 2024-04-25T16:30:22.990 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 16:24:38.733786331 +0000 2024-04-25T16:30:22.990 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 16:24:38.733786331 +0000 2024-04-25T16:30:22.990 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 16:24:38.733786331 +0000 2024-04-25T16:30:22.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:30:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T16:30:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T16:30:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000199349 s, 2.6 MB/s 2024-04-25T16:30:23.054 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:30:23.110 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 16:29:27.235665620 +0000 2024-04-25T16:30:23.166 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 16:24:39.068776653 +0000 2024-04-25T16:30:23.167 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 16:24:39.068776653 +0000 2024-04-25T16:30:23.167 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 16:24:39.068776653 +0000 2024-04-25T16:30:23.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:30:23.228 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T16:30:23.228 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T16:30:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000173935 s, 2.9 MB/s 2024-04-25T16:30:23.229 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:30:23.284 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 16:29:27.266664784 +0000 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 16:24:39.404766947 +0000 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 16:24:39.404766947 +0000 2024-04-25T16:30:23.339 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 16:24:39.404766947 +0000 2024-04-25T16:30:23.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:30:23.400 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T16:30:23.401 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T16:30:23.401 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00019315 s, 2.7 MB/s 2024-04-25T16:30:23.402 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:30:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi116=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: Saving service mgr spec with placement smithi112=x;smithi116=y;smithi133=z;count:3 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:23.457 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-04-25T16:30:23.457 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:30:23.476 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:30:23.476 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-04-25T16:30:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi116=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: Saving service mgr spec with placement smithi112=x;smithi116=y;smithi133=z;count:3 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-04-25 16:30:07.785409984 +0000 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-04-25 16:24:56.365123584 +0000 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-04-25 16:24:56.365123584 +0000 2024-04-25T16:30:23.527 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-04-25 16:24:56.365123584 +0000 2024-04-25T16:30:23.528 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:30:23.593 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-04-25T16:30:23.593 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-04-25T16:30:23.594 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000298804 s, 1.7 MB/s 2024-04-25T16:30:23.595 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:30:23.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi112=x;smithi116=y;smithi133=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:23.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: Saving service mgr spec with placement smithi112=x;smithi116=y;smithi133=z;count:3 2024-04-25T16:30:23.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:23.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:23.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:23.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:23.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:23.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T16:30:23.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:23.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:23.653 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-04-25 16:30:07.785409984 +0000 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-04-25 16:24:56.697117968 +0000 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-04-25 16:24:56.697117968 +0000 2024-04-25T16:30:23.710 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-04-25 16:24:56.697117968 +0000 2024-04-25T16:30:23.710 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:30:23.775 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-04-25T16:30:23.775 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-04-25T16:30:23.775 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.00022924 s, 2.2 MB/s 2024-04-25T16:30:23.777 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:30:23.834 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-04-25T16:30:23.889 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-04-25 16:30:07.786409969 +0000 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-04-25 16:24:57.016112573 +0000 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-04-25 16:24:57.016112573 +0000 2024-04-25T16:30:23.890 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-04-25 16:24:57.016112573 +0000 2024-04-25T16:30:23.890 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:30:23.952 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-04-25T16:30:23.952 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-04-25T16:30:23.952 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000253775 s, 2.0 MB/s 2024-04-25T16:30:23.954 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:30:24.010 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-04-25T16:30:24.065 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:30:24.065 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:24.065 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T16:30:24.066 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:24.066 INFO:teuthology.orchestra.run.smithi116.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:24.066 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-04-25 16:30:07.786409969 +0000 2024-04-25T16:30:24.066 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-04-25 16:24:57.355106839 +0000 2024-04-25T16:30:24.066 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-04-25 16:24:57.355106839 +0000 2024-04-25T16:30:24.066 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: 2024-04-25 16:24:57.355106839 +0000 2024-04-25T16:30:24.066 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:30:24.127 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-04-25T16:30:24.128 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-04-25T16:30:24.128 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000249872 s, 2.0 MB/s 2024-04-25T16:30:24.129 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:30:24.186 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-04-25T16:30:24.186 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T16:30:24.203 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T16:30:24.203 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 16:30:24.069663845 +0000 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 16:24:55.640668652 +0000 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 16:24:55.640668652 +0000 2024-04-25T16:30:24.259 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-25 16:24:55.640668652 +0000 2024-04-25T16:30:24.260 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T16:30:24.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:24 smithi116 ceph-mon[32712]: Deploying daemon mgr.z on smithi133 2024-04-25T16:30:24.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:24 smithi116 ceph-mon[32712]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T16:30:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T16:30:24.323 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000254158 s, 2.0 MB/s 2024-04-25T16:30:24.324 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T16:30:24.379 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-04-25T16:30:24.434 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T16:30:24.434 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:24.434 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T16:30:24.434 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:24.434 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:24.434 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 16:30:24.069663845 +0000 2024-04-25T16:30:24.435 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 16:24:55.988661757 +0000 2024-04-25T16:30:24.435 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 16:24:55.988661757 +0000 2024-04-25T16:30:24.435 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-25 16:24:55.988661757 +0000 2024-04-25T16:30:24.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T16:30:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:24 smithi133 ceph-mon[32691]: Deploying daemon mgr.z on smithi133 2024-04-25T16:30:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:24 smithi133 ceph-mon[32691]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:24.451 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 podman[34358]: 2024-04-25 16:30:24.374468052 +0000 UTC m=+0.143962428 container create b0b3a1821e60261d6092dbe13ecddb7a92607f16c7b998eda47a39112d5091a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T16:30:24.472 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T16:30:24.472 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T16:30:24.472 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000200445 s, 2.6 MB/s 2024-04-25T16:30:24.473 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T16:30:24.532 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 16:30:24.070663826 +0000 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 16:24:56.330654981 +0000 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 16:24:56.330654981 +0000 2024-04-25T16:30:24.584 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-25 16:24:56.330654981 +0000 2024-04-25T16:30:24.584 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T16:30:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:24 smithi112 ceph-mon[29122]: Deploying daemon mgr.z on smithi133 2024-04-25T16:30:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:24 smithi112 ceph-mon[29122]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:24.647 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T16:30:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T16:30:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000187151 s, 2.7 MB/s 2024-04-25T16:30:24.649 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T16:30:24.705 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-04-25 16:30:24.070663826 +0000 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-04-25 16:24:56.655648541 +0000 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-04-25 16:24:56.655648541 +0000 2024-04-25T16:30:24.761 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-04-25 16:24:56.655648541 +0000 2024-04-25T16:30:24.761 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T16:30:24.798 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 podman[34358]: 2024-04-25 16:30:24.627063437 +0000 UTC m=+0.396557823 container init b0b3a1821e60261d6092dbe13ecddb7a92607f16c7b998eda47a39112d5091a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T16:30:24.798 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 podman[34358]: 2024-04-25 16:30:24.629962819 +0000 UTC m=+0.399457204 container start b0b3a1821e60261d6092dbe13ecddb7a92607f16c7b998eda47a39112d5091a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T16:30:24.798 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 ceph-mgr[34501]: -- 172.21.15.133:0/2271364488 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55888426c5a0 con 0x558888ff6000 2024-04-25T16:30:24.798 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 bash[34358]: b0b3a1821e60261d6092dbe13ecddb7a92607f16c7b998eda47a39112d5091a1 2024-04-25T16:30:24.798 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 systemd[1]: Started Ceph mgr.z for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:30:24.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-04-25T16:30:24.820 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-04-25T16:30:24.821 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000175912 s, 2.9 MB/s 2024-04-25T16:30:24.822 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T16:30:24.878 INFO:tasks.cephadm:Deploying osd.0 on smithi112 with /dev/vg_nvme/lv_4... 2024-04-25T16:30:24.878 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T16:30:25.090 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:24.798+0000 7fdda0605200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:30:25.090 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:24.988+0000 7fdda0605200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:30:25.090 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:25.090+0000 7fdda0605200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:30:25.116 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:30:25.700 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:25.398+0000 7fdda0605200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:30:25.700 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:25.481+0000 7fdda0605200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:30:25.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: Deploying daemon mgr.y on smithi116 2024-04-25T16:30:25.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:25 smithi112 ceph-mon[29122]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:26.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:26.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:26.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:26.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:26.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T16:30:26.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:26.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:26.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: Deploying daemon mgr.y on smithi116 2024-04-25T16:30:26.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:25 smithi116 ceph-mon[32712]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: Deploying daemon mgr.y on smithi116 2024-04-25T16:30:26.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:25 smithi133 ceph-mon[32691]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:26.759 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:26 smithi116 systemd[1]: Starting Ceph mgr.y for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:30:26.950 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:26.672+0000 7fdda0605200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:30:26.951 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:26.841+0000 7fdda0605200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:30:26.957 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T16:30:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T16:30:26.958 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T16:30:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T16:30:26.958 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116033 s, 904 MB/s 2024-04-25T16:30:26.958 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T16:30:27.172 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:26 smithi116 podman[34080]: 2024-04-25 16:30:26.80340048 +0000 UTC m=+0.018206294 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:30:27.172 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:26 smithi116 podman[34080]: 2024-04-25 16:30:26.907187971 +0000 UTC m=+0.121993824 container create 77c04953f41c5bbbf0a7b8f83ad2d0f6bb8cbc57ac78f502df0574d9dd8cdddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:30:27.172 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 podman[34080]: 2024-04-25 16:30:27.13451032 +0000 UTC m=+0.349316127 container init 77c04953f41c5bbbf0a7b8f83ad2d0f6bb8cbc57ac78f502df0574d9dd8cdddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T16:30:27.172 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 podman[34080]: 2024-04-25 16:30:27.137400593 +0000 UTC m=+0.352206401 container start 77c04953f41c5bbbf0a7b8f83ad2d0f6bb8cbc57ac78f502df0574d9dd8cdddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T16:30:27.172 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 ceph-mgr[34099]: -- 172.21.15.116:0/1729913328 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5595ffcba5a0 con 0x559604a44480 2024-04-25T16:30:27.321 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:27.019+0000 7fdda0605200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:30:27.321 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:27.321+0000 7fdda0605200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:30:27.450 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 bash[34080]: 77c04953f41c5bbbf0a7b8f83ad2d0f6bb8cbc57ac78f502df0574d9dd8cdddb 2024-04-25T16:30:27.451 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 systemd[1]: Started Ceph mgr.y for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:30:27.451 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:27.308+0000 7f021bf5b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:30:27.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_4 2024-04-25T16:30:27.632 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:27.418+0000 7fdda0605200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:30:27.632 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:27.632+0000 7fdda0605200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:30:27.759 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:27.503+0000 7f021bf5b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:30:27.759 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:27.614+0000 7f021bf5b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:30:27.847 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:30:27.950 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:27.834+0000 7fdda0605200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:27 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:27.940+0000 7f021bf5b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:28.034+0000 7f021bf5b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-mon[32712]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:28.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:28 smithi112 ceph-mon[29122]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:28.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.261 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:28.331 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-mon[32691]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:28.331 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.331 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.331 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.331 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:28.331 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:28.586 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:28.507+0000 7fdda0605200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:30:29.257 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:28.959+0000 7fdda0605200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:30:29.257 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:29.058+0000 7fdda0605200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:30:29.257 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:29.257+0000 7fdda0605200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:30:29.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 -- 172.21.15.112:0/2330781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 msgr2=0x7f04ec106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:29.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 --2- 172.21.15.112:0/2330781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec106d50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009990 tx=0x7f04dc009ca0 comp rx=0 tx=0).stop 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 -- 172.21.15.112:0/2330781930 shutdown_connections 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 --2- 172.21.15.112:0/2330781930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ec109e90 0x7f04ec10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 --2- 172.21.15.112:0/2330781930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec106d50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 --2- 172.21.15.112:0/2330781930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 0x7f04ec109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 -- 172.21.15.112:0/2330781930 >> 172.21.15.112:0/2330781930 conn(0x7f04ec0fe740 msgr2=0x7f04ec100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 -- 172.21.15.112:0/2330781930 shutdown_connections 2024-04-25T16:30:29.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.426+0000 7f04f0925700 1 -- 172.21.15.112:0/2330781930 wait complete. 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 Processor -- start 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 -- start start 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 0x7f04ec19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ec109e90 0x7f04ec1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec10fc60 con 0x7f04ec06c1a0 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec10fae0 con 0x7f04ec107320 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.427+0000 7f04f0925700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04ec10fde0 con 0x7f04ec109e90 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04ea59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04ea59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40292/0 (socket says 172.21.15.112:40292) 2024-04-25T16:30:29.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04ea59c700 1 -- 172.21.15.112:0/1921512810 learned_addr learned my addr 172.21.15.112:0/1921512810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 0x7f04ec19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04ead9d700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ec109e90 0x7f04ec1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 -- 172.21.15.112:0/1921512810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ec109e90 msgr2=0x7f04ec1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ec109e90 0x7f04ec1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 -- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 msgr2=0x7f04ec19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 -- 172.21.15.112:0/1921512810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ec1a22d0 con 0x7f04ec107320 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04ea59c700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04e9d9b700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 0x7f04ec19acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f04e000e9f0 tx=0x7f04e000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e000c9c0 con 0x7f04ec107320 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.428+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e000cb60 con 0x7f04ec107320 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.429+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0018680 con 0x7f04ec107320 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.429+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ec1a25c0 con 0x7f04ec107320 2024-04-25T16:30:29.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.429+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ec1a2ab0 con 0x7f04ec107320 2024-04-25T16:30:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.430+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 13) ==== 49573+0+0 (secure 0 0 0) 0x7f04e0018820 con 0x7f04ec107320 2024-04-25T16:30:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.430+0000 7f04db7fe700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f04d403d850 0x7f04d403fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:29.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.430+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f04e00518a0 con 0x7f04ec107320 2024-04-25T16:30:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.430+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ec051190 con 0x7f04ec107320 2024-04-25T16:30:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.430+0000 7f04ea59c700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f04d403d850 0x7f04d403fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:29.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.430+0000 7f04ea59c700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f04d403d850 0x7f04d403fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00b2e0 tx=0x7f04dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:29.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.436+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04e0015dc0 con 0x7f04ec107320 2024-04-25T16:30:29.610 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:29.274+0000 7f021bf5b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:30:29.610 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:29.443+0000 7f021bf5b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:30:29.610 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:29.610+0000 7f021bf5b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:30:29.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:29.653+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f04ec063ad0 con 0x7f04d403d850 2024-04-25T16:30:30.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: Reconfiguring daemon mgr.x on smithi112 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:30:30.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.111 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:29.778+0000 7fdda0605200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:30:30.111 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:29.870+0000 7fdda0605200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:30:30.111 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: Reconfiguring daemon mgr.x on smithi112 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:30:30.112 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:30:30.113 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:29.896+0000 7f021bf5b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:29.988+0000 7f021bf5b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:30.190+0000 7f021bf5b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.190 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: Reconfiguring daemon mgr.x on smithi112 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:30:30.191 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:30.450 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:30.111+0000 7fdda0605200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:30:30.451 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:30.243+0000 7fdda0605200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:30:30.451 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:30.337+0000 7fdda0605200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:30:30.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:30.383+0000 7f021bf5b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:30:30.758 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:30.452+0000 7fdda0605200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:30:30.758 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:30.758+0000 7fdda0605200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:30:31.259 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:30.984+0000 7f021bf5b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:31.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:31.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:31.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:31.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:31.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.355 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:31.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:31.698 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:31.405+0000 7f021bf5b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:30:31.698 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:31.500+0000 7f021bf5b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:30:31.698 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:31 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:31.698+0000 7f021bf5b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:30:31.700 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:31.356+0000 7fdda0605200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:30:31.700 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:30:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:30:31.570+0000 7fdda0605200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:30:32.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:32.060+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 7 ==== mgrmap(e 14) ==== 98519+0+0 (secure 0 0 0) 0x7f04e00173d0 con 0x7f04ec107320 2024-04-25T16:30:32.308 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:32.217+0000 7f021bf5b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:30:32.308 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:32.308+0000 7f021bf5b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:30:32.308 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-mon[32712]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:32.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-mon[32712]: Standby manager daemon z started 2024-04-25T16:30:32.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T16:30:32.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:30:32.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T16:30:32.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:30:32.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:32 smithi112 ceph-mon[29122]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:32.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:32 smithi112 ceph-mon[29122]: Standby manager daemon z started 2024-04-25T16:30:32.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:32 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T16:30:32.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:32 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:30:32.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:32 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T16:30:32.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:32 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:30:32.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:32 smithi133 ceph-mon[32691]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:32.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:32 smithi133 ceph-mon[32691]: Standby manager daemon z started 2024-04-25T16:30:32.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:32 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T16:30:32.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:32 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:30:32.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:32 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T16:30:32.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:32 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1000594547' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:30:32.679 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:32.548+0000 7f021bf5b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:30:32.680 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:32.679+0000 7f021bf5b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:30:33.009 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:32.774+0000 7f021bf5b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:30:33.009 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:32.891+0000 7f021bf5b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:30:33.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:33 smithi112 ceph-mon[29122]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T16:30:33.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T16:30:33.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:33 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2285585632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2011d44b-d406-413a-aa94-1a3a50fcde37"}]: dispatch 2024-04-25T16:30:33.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:33 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2285585632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2011d44b-d406-413a-aa94-1a3a50fcde37"}]': finished 2024-04-25T16:30:33.223 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:33 smithi112 ceph-mon[29122]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T16:30:33.224 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:33.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:33 smithi133 ceph-mon[32691]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T16:30:33.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T16:30:33.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:33 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2285585632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2011d44b-d406-413a-aa94-1a3a50fcde37"}]: dispatch 2024-04-25T16:30:33.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:33 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2285585632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2011d44b-d406-413a-aa94-1a3a50fcde37"}]': finished 2024-04-25T16:30:33.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:33 smithi133 ceph-mon[32691]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T16:30:33.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:33.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:33.195+0000 7f021bf5b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:30:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-mon[32712]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T16:30:33.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T16:30:33.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2285585632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2011d44b-d406-413a-aa94-1a3a50fcde37"}]: dispatch 2024-04-25T16:30:33.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2285585632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2011d44b-d406-413a-aa94-1a3a50fcde37"}]': finished 2024-04-25T16:30:33.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-mon[32712]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T16:30:33.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:34.058 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:33 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:33.791+0000 7f021bf5b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:30:34.059 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:30:34.006+0000 7f021bf5b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:30:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3019487609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:30:34.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: Standby manager daemon y started 2024-04-25T16:30:34.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T16:30:34.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:30:34.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T16:30:34.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:34 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:30:34.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:34.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3019487609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:30:34.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: Standby manager daemon y started 2024-04-25T16:30:34.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T16:30:34.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:30:34.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T16:30:34.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:34 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:30:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3019487609' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:30:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: Standby manager daemon y started 2024-04-25T16:30:34.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T16:30:34.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:30:34.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T16:30:34.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:34 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/2183709497' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:30:34.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:34.840+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mon.2 v2:172.21.15.116:3300/0 8 ==== mgrmap(e 15) ==== 147465+0+0 (secure 0 0 0) 0x7f04e001d210 con 0x7f04ec107320 2024-04-25T16:30:36.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:35 smithi112 ceph-mon[29122]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T16:30:36.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:35 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T16:30:36.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:35 smithi112 ceph-mon[29122]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:36.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:35 smithi133 ceph-mon[32691]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T16:30:36.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:35 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T16:30:36.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:35 smithi133 ceph-mon[32691]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:36.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:35 smithi116 ceph-mon[32712]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T16:30:36.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:35 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T16:30:36.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:35 smithi116 ceph-mon[32712]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:38.199 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:38 smithi112 ceph-mon[29122]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:38.383 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:38 smithi116 ceph-mon[32712]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:38.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:38 smithi133 ceph-mon[32691]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:40.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:40 smithi112 ceph-mon[29122]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:40.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:40 smithi133 ceph-mon[32691]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:40 smithi116 ceph-mon[32712]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T16:30:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:41 smithi112 ceph-mon[29122]: Deploying daemon osd.0 on smithi112 2024-04-25T16:30:41.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T16:30:41.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:41.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:41 smithi133 ceph-mon[32691]: Deploying daemon osd.0 on smithi112 2024-04-25T16:30:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T16:30:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:41 smithi116 ceph-mon[32712]: Deploying daemon osd.0 on smithi112 2024-04-25T16:30:42.339 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:42 smithi112 ceph-mon[29122]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:42.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:42 smithi133 ceph-mon[32691]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:42.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:42 smithi116 ceph-mon[32712]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:44.248 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:44 smithi112 ceph-mon[29122]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:44.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:44 smithi133 ceph-mon[32691]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:44.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:44 smithi116 ceph-mon[32712]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:46 smithi112 ceph-mon[29122]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:46.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:46.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:46.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:46 smithi133 ceph-mon[32691]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:46.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:46 smithi116 ceph-mon[32712]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:48.330 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:48 smithi112 ceph-mon[29122]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:48.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:48 smithi133 ceph-mon[32691]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:48 smithi116 ceph-mon[32712]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:49.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.562+0000 7f04db7fe700 1 -- 172.21.15.112:0/1921512810 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f04ec063ad0 con 0x7f04d403d850 2024-04-25T16:30:49.563 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 0 on host 'smithi112' 2024-04-25T16:30:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.564+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f04d403d850 msgr2=0x7f04d403fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.564+0000 7f04f0925700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f04d403d850 0x7f04d403fd60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00b2e0 tx=0x7f04dc005fb0 comp rx=0 tx=0).stop 2024-04-25T16:30:49.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.564+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 msgr2=0x7f04ec19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.564+0000 7f04f0925700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 0x7f04ec19acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f04e000e9f0 tx=0x7f04e000ed00 comp rx=0 tx=0).stop 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.564+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 shutdown_connections 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04ec109e90 0x7f04ec1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04ec06c1a0 0x7f04ec19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f04d403d850 0x7f04d403fd60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 --2- 172.21.15.112:0/1921512810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04ec107320 0x7f04ec19acb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:49.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 >> 172.21.15.112:0/1921512810 conn(0x7f04ec0fe740 msgr2=0x7f04ec1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 shutdown_connections 2024-04-25T16:30:49.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:30:49.565+0000 7f04f0925700 1 -- 172.21.15.112:0/1921512810 wait complete. 2024-04-25T16:30:49.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:49.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:49.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:49.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:49.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:49.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:49 smithi112 ceph-mon[29122]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:49.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:49.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:49.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:49.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:49.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:49.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:49 smithi133 ceph-mon[32691]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:50.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:50.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:50.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:49 smithi116 ceph-mon[32712]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:50.289 DEBUG:teuthology.orchestra.run.smithi112:osd.0> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T16:30:50.291 INFO:tasks.cephadm:Deploying osd.1 on smithi116 with /dev/vg_nvme/lv_4... 2024-04-25T16:30:50.291 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T16:30:50.533 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:30:50.655 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.655 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.655 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:50.655 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:50 smithi116 ceph-mon[32712]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T16:30:50.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:50.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:50 smithi112 ceph-mon[29122]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T16:30:50.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:50.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:30:50.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:50 smithi133 ceph-mon[32691]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T16:30:51.636 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:51 smithi116 ceph-mon[32712]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T16:30:51.637 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:51 smithi116 ceph-mon[32712]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T16:30:51.637 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:51 smithi116 ceph-mon[32712]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:30:51.637 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:51 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:51.637 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:51 smithi116 ceph-mon[32712]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:51.801 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-mon[29122]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T16:30:51.802 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-mon[29122]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T16:30:51.802 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-mon[29122]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:30:51.802 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:51.802 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-mon[29122]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:51.802 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0[44345]: 2024-04-25T16:30:51.553+0000 7fb3600f2700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T16:30:51.802 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:30:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0[44345]: 2024-04-25T16:30:51.558+0000 7fb35bea7700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:30:51.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:51 smithi133 ceph-mon[32691]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T16:30:51.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:51 smithi133 ceph-mon[32691]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T16:30:51.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:51 smithi133 ceph-mon[32691]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:30:51.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:51 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:51.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:51 smithi133 ceph-mon[32691]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:52.527 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T16:30:52.527 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T16:30:52.527 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-04-25T16:30:52.527 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-04-25T16:30:52.528 INFO:teuthology.orchestra.run.smithi116.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011358 s, 923 MB/s 2024-04-25T16:30:52.528 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 2024-04-25T16:30:52.528 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-04-25T16:30:52.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:52 smithi112 ceph-mon[29122]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:30:52.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:52 smithi112 ceph-mon[29122]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T16:30:52.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:52.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:52.886 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:52 smithi116 ceph-mon[32712]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:30:52.886 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:52 smithi116 ceph-mon[32712]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T16:30:52.886 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:52.887 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:52.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:52 smithi133 ceph-mon[32691]: from='osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:30:52.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:52 smithi133 ceph-mon[32691]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T16:30:52.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:52.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.206 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_4 2024-04-25T16:30:53.443 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:30:53.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] boot 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:53.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:30:53.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:30:53.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] boot 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:53.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: osd.0 [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] boot 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:30:53.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:30:54.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:54 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi112 2024-04-25T16:30:54.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:54 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T16:30:54.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:54 smithi116 ceph-mon[32712]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 -- 172.21.15.116:0/1959083557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 msgr2=0x7f10dc10b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1959083557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc10b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f10d400b0a0 tx=0x7f10d400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 -- 172.21.15.116:0/1959083557 shutdown_connections 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1959083557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc10b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1959083557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10dc103100 0x7f10dc103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1959083557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc104230 0x7f10dc104640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 -- 172.21.15.116:0/1959083557 >> 172.21.15.116:0/1959083557 conn(0x7f10dc0fe740 msgr2=0x7f10dc100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 -- 172.21.15.116:0/1959083557 shutdown_connections 2024-04-25T16:30:54.842 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.842+0000 7f10e45e5700 1 -- 172.21.15.116:0/1959083557 wait complete. 2024-04-25T16:30:54.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 Processor -- start 2024-04-25T16:30:54.844 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 -- start start 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc103100 0x7f10dc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10dc104230 0x7f10dc199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc10eb70 con 0x7f10dc103100 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc10e9f0 con 0x7f10dc104bd0 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e45e5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc10ecf0 con 0x7f10dc104230 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc1a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48904/0 (socket says 172.21.15.116:48904) 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 -- 172.21.15.116:0/1584678703 learned_addr learned my addr 172.21.15.116:0/1584678703 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2381700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc103100 0x7f10dc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e1b80700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10dc104230 0x7f10dc199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 -- 172.21.15.116:0/1584678703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10dc104230 msgr2=0x7f10dc199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10dc104230 0x7f10dc199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:54.845 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 -- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc103100 msgr2=0x7f10dc199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.843+0000 7f10e2b82700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc103100 0x7f10dc199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10e2b82700 1 -- 172.21.15.116:0/1584678703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1a1160 con 0x7f10dc104bd0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10e2b82700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc1a0ae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10d40062a0 tx=0x7f10d4004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4021070 con 0x7f10dc104bd0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc1a13f0 con 0x7f10dc104bd0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10d4007b80 con 0x7f10dc104bd0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4004670 con 0x7f10dc104bd0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.844+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1a1930 con 0x7f10dc104bd0 2024-04-25T16:30:54.846 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.845+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc051190 con 0x7f10dc104bd0 2024-04-25T16:30:54.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:54 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi112 2024-04-25T16:30:54.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:54 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T16:30:54.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:54 smithi112 ceph-mon[29122]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T16:30:54.847 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.846+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 15) ==== 147465+0+0 (secure 0 0 0) 0x7f10d4003a00 con 0x7f10dc104bd0 2024-04-25T16:30:54.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.847+0000 7f10d37fe700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f10c80afaf0 0x7f10c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:30:54.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.847+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f10d400f020 con 0x7f10dc104bd0 2024-04-25T16:30:54.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.847+0000 7f10e2381700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f10c80afaf0 0x7f10c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:30:54.848 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.847+0000 7f10e2381700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f10c80afaf0 0x7f10c80b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10dc104130 tx=0x7f10d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:30:54.851 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:54.852+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10d402a020 con 0x7f10dc104bd0 2024-04-25T16:30:54.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:54 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi112 2024-04-25T16:30:54.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:54 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi112 to 17088M 2024-04-25T16:30:54.851 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:54 smithi133 ceph-mon[32691]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T16:30:55.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:30:55.057+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f10dc0758c0 con 0x7f10c80afaf0 2024-04-25T16:30:55.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:55 smithi112 ceph-mon[29122]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:30:55.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:55 smithi112 ceph-mon[29122]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:55.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:30:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:30:55.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:55.881 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:55 smithi116 ceph-mon[32712]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:30:55.881 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:55 smithi116 ceph-mon[32712]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:55.881 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:30:55.881 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:30:55.881 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:55.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:55 smithi133 ceph-mon[32691]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:30:55.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:55 smithi133 ceph-mon[32691]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:30:55.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:30:55.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:30:55.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:30:58.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:58 smithi116 ceph-mon[32712]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:30:58.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:58 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/129592252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]: dispatch 2024-04-25T16:30:58.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:58 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]: dispatch 2024-04-25T16:30:58.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:58 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]': finished 2024-04-25T16:30:58.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:58 smithi116 ceph-mon[32712]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T16:30:58.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:30:58.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:58 smithi112 ceph-mon[29122]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:30:58.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:58 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/129592252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]: dispatch 2024-04-25T16:30:58.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:58 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]: dispatch 2024-04-25T16:30:58.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:58 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]': finished 2024-04-25T16:30:58.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:58 smithi112 ceph-mon[29122]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T16:30:58.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:30:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:58 smithi133 ceph-mon[32691]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:30:58.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:58 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/129592252' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]: dispatch 2024-04-25T16:30:58.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:58 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]: dispatch 2024-04-25T16:30:58.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:58 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb335e86-8675-4ab0-826c-9cf84a05a605"}]': finished 2024-04-25T16:30:58.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:58 smithi133 ceph-mon[32691]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T16:30:58.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:30:59.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:30:59 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/1890032837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:30:59.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:30:59 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/1890032837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:30:59.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:30:59 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/1890032837' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:00.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:00 smithi112 ceph-mon[29122]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:00 smithi133 ceph-mon[32691]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:00.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:00 smithi116 ceph-mon[32712]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:02.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:02 smithi112 ceph-mon[29122]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:02 smithi133 ceph-mon[32691]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:02.493 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:02 smithi116 ceph-mon[32712]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:04.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:04 smithi116 ceph-mon[32712]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:04.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:04 smithi112 ceph-mon[29122]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:04 smithi133 ceph-mon[32691]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:05.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T16:31:05.334 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:05.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T16:31:05.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:05.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T16:31:05.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:06.233 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:06 smithi116 ceph-mon[32712]: Deploying daemon osd.1 on smithi116 2024-04-25T16:31:06.233 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:06 smithi116 ceph-mon[32712]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:06 smithi112 ceph-mon[29122]: Deploying daemon osd.1 on smithi116 2024-04-25T16:31:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:06 smithi112 ceph-mon[29122]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:06 smithi133 ceph-mon[32691]: Deploying daemon osd.1 on smithi116 2024-04-25T16:31:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:06 smithi133 ceph-mon[32691]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:08.149 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:08 smithi116 ceph-mon[32712]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:08 smithi112 ceph-mon[29122]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:08 smithi133 ceph-mon[32691]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:10.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:10 smithi116 ceph-mon[32712]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:10.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:10.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:10.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:10.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:10 smithi112 ceph-mon[29122]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:10.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:10.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:10.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:10 smithi133 ceph-mon[32691]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:10.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:10.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:10.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:12 smithi112 ceph-mon[29122]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:12.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:12 smithi116 ceph-mon[32712]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:12.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:12 smithi133 ceph-mon[32691]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:13.569 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 1 on host 'smithi116' 2024-04-25T16:31:13.570 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.569+0000 7f10d37fe700 1 -- 172.21.15.116:0/1584678703 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f10dc0758c0 con 0x7f10c80afaf0 2024-04-25T16:31:13.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f10c80afaf0 msgr2=0x7f10c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:13.571 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f10c80afaf0 0x7f10c80b2000 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10dc104130 tx=0x7f10d800b410 comp rx=0 tx=0).stop 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 msgr2=0x7f10dc1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc1a0ae0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10d40062a0 tx=0x7f10d4004e40 comp rx=0 tx=0).stop 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 shutdown_connections 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f10c80afaf0 0x7f10c80b2000 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10dc104bd0 0x7f10dc1a0ae0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10dc103100 0x7f10dc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 --2- 172.21.15.116:0/1584678703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10dc104230 0x7f10dc199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.572+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 >> 172.21.15.116:0/1584678703 conn(0x7f10dc0fe740 msgr2=0x7f10dc107460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:31:13.572 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.573+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 shutdown_connections 2024-04-25T16:31:13.573 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:31:13.573+0000 7f10e45e5700 1 -- 172.21.15.116:0/1584678703 wait complete. 2024-04-25T16:31:13.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:13.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:13.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:13.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:13.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:13.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:13.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:13.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:13.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:14.187 DEBUG:teuthology.orchestra.run.smithi116:osd.1> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T16:31:14.189 INFO:tasks.cephadm:Deploying osd.2 on smithi133 with /dev/vg_nvme/lv_4... 2024-04-25T16:31:14.189 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T16:31:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:31:14.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:14 smithi116 ceph-mon[32712]: from='osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T16:31:14.941 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:14 smithi116 ceph-mon[32712]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T16:31:14.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:14 smithi133 ceph-mon[32691]: from='osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T16:31:14.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:14 smithi133 ceph-mon[32691]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T16:31:15.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:14 smithi112 ceph-mon[29122]: from='osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T16:31:15.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:14 smithi112 ceph-mon[29122]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T16:31:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:15 smithi133 ceph-mon[32691]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T16:31:15.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:15 smithi133 ceph-mon[32691]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T16:31:15.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:15.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:15 smithi133 ceph-mon[32691]: from='osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:31:15.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:15 smithi133 ceph-mon[32691]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:31:15.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:15 smithi133 ceph-mon[32691]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-mon[32712]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T16:31:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-mon[32712]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T16:31:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:16.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-mon[32712]: from='osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:31:16.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-mon[32712]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:31:16.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-mon[32712]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:16.010 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1[37293]: 2024-04-25T16:31:15.623+0000 7fe6b694d700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T16:31:16.010 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1[37293]: 2024-04-25T16:31:15.628+0000 7fe6b2702700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:31:16.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:15 smithi112 ceph-mon[29122]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T16:31:16.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:15 smithi112 ceph-mon[29122]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T16:31:16.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:16.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:15 smithi112 ceph-mon[29122]: from='osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:31:16.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:15 smithi112 ceph-mon[29122]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:31:16.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:15 smithi112 ceph-mon[29122]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:16.161 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T16:31:16.161 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T16:31:16.161 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-25T16:31:16.161 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-25T16:31:16.161 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112193 s, 935 MB/s 2024-04-25T16:31:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-25T16:31:16.731 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:16 smithi116 ceph-mon[32712]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:31:16.731 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:16 smithi116 ceph-mon[32712]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T16:31:16.732 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:16.732 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:16.786 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:16 smithi133 ceph-mon[32691]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:31:16.786 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:16 smithi133 ceph-mon[32691]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T16:31:16.786 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:16.786 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:16.814 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2024-04-25T16:31:17.055 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:31:17.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:16 smithi112 ceph-mon[29122]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:31:17.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:16 smithi112 ceph-mon[29122]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T16:31:17.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:17.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] boot 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi116 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:17.858 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:17.859 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:31:17.859 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi116 to 17088M 2024-04-25T16:31:17.859 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:17.859 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:17.859 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:17.859 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:31:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] boot 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi116 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi116 to 17088M 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:18.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:31:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:31:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: osd.1 [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] boot 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi116 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi116 to 17088M 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:18.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 -- 172.21.15.133:0/1716217439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de806cbd0 msgr2=0x7f8de8106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 --2- 172.21.15.133:0/1716217439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de806cbd0 0x7f8de8106d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8de400b0a0 tx=0x7f8de400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:31:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 -- 172.21.15.133:0/1716217439 shutdown_connections 2024-04-25T16:31:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 --2- 172.21.15.133:0/1716217439 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8109eb0 0x7f8de810c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:18.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 --2- 172.21.15.133:0/1716217439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de806cbd0 0x7f8de8106d70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 --2- 172.21.15.133:0/1716217439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de8107340 0x7f8de8109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.600+0000 7f8df0385700 1 -- 172.21.15.133:0/1716217439 >> 172.21.15.133:0/1716217439 conn(0x7f8de80fe740 msgr2=0x7f8de8100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 -- 172.21.15.133:0/1716217439 shutdown_connections 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 -- 172.21.15.133:0/1716217439 wait complete. 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 Processor -- start 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 -- start start 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de806cbd0 0x7f8de819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8107340 0x7f8de819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 0x7f8de81a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8de810fc50 con 0x7f8de8107340 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8de810fad0 con 0x7f8de806cbd0 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8df0385700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8de810fdd0 con 0x7f8de8109eb0 2024-04-25T16:31:18.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.601+0000 7f8dee922700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 0x7f8de81a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 0x7f8de81a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52576/0 (socket says 172.21.15.133:52576) 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 -- 172.21.15.133:0/3619613365 learned_addr learned my addr 172.21.15.133:0/3619613365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee121700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de806cbd0 0x7f8de819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8ded920700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8107340 0x7f8de819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 -- 172.21.15.133:0/3619613365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de806cbd0 msgr2=0x7f8de819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de806cbd0 0x7f8de819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 -- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8107340 msgr2=0x7f8de819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8107340 0x7f8de819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 -- 172.21.15.133:0/3619613365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de81a22e0 con 0x7f8de8109eb0 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee121700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de806cbd0 0x7f8de819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8ded920700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8107340 0x7f8de819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8dee922700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 0x7f8de81a1c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8de000d980 tx=0x7f8de000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0016070 con 0x7f8de8109eb0 2024-04-25T16:31:18.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de81a25d0 con 0x7f8de8109eb0 2024-04-25T16:31:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de0007e20 con 0x7f8de8109eb0 2024-04-25T16:31:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0004d30 con 0x7f8de8109eb0 2024-04-25T16:31:18.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.602+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de81a2b10 con 0x7f8de8109eb0 2024-04-25T16:31:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.603+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de8051190 con 0x7f8de8109eb0 2024-04-25T16:31:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.604+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) ==== 147465+0+0 (secure 0 0 0) 0x7f8de000bb60 con 0x7f8de8109eb0 2024-04-25T16:31:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.605+0000 7f8ddf7fe700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8dd40afaf0 0x7f8dd40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.605+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8de00dd200 con 0x7f8de8109eb0 2024-04-25T16:31:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.605+0000 7f8dee121700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8dd40afaf0 0x7f8dd40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:18.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.606+0000 7f8dee121700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8dd40afaf0 0x7f8dd40b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009e60 tx=0x7f8de4009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:31:18.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.610+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8de001f020 con 0x7f8de8109eb0 2024-04-25T16:31:18.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:18.817+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8de8063ad0 con 0x7f8dd40afaf0 2024-04-25T16:31:18.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:18 smithi133 ceph-mon[32691]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T16:31:19.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:18 smithi116 ceph-mon[32712]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T16:31:19.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:18 smithi112 ceph-mon[29122]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T16:31:19.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:19 smithi133 ceph-mon[32691]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:31:19.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:31:19.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:31:19.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:19.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:19 smithi133 ceph-mon[32691]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:20.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:19 smithi116 ceph-mon[32712]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:31:20.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:31:20.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:31:20.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:20.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:19 smithi116 ceph-mon[32712]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:20.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:19 smithi112 ceph-mon[29122]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:31:20.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:31:20.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:31:20.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:20.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:19 smithi112 ceph-mon[29122]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/2155301332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]: dispatch 2024-04-25T16:31:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]: dispatch 2024-04-25T16:31:22.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]': finished 2024-04-25T16:31:22.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T16:31:22.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:22.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/1176064038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/2155301332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]: dispatch 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]: dispatch 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]': finished 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:22.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:22 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/1176064038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/2155301332' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]: dispatch 2024-04-25T16:31:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]: dispatch 2024-04-25T16:31:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a075e9d7-1972-4815-8471-6a49816ec39f"}]': finished 2024-04-25T16:31:22.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T16:31:22.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:22.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:22 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/1176064038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:24.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:24 smithi112 ceph-mon[29122]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:24 smithi133 ceph-mon[32691]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:24 smithi116 ceph-mon[32712]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:26.322 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:26 smithi133 ceph-mon[32691]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:26.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:26 smithi116 ceph-mon[32712]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:26.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:26 smithi112 ceph-mon[29122]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:28.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:28 smithi112 ceph-mon[29122]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:28 smithi133 ceph-mon[32691]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:28 smithi116 ceph-mon[32712]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:29.165 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T16:31:29.166 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T16:31:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:29.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T16:31:29.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:30 smithi112 ceph-mon[29122]: Deploying daemon osd.2 on smithi133 2024-04-25T16:31:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:30 smithi112 ceph-mon[29122]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:30.411 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:30 smithi133 ceph-mon[32691]: Deploying daemon osd.2 on smithi133 2024-04-25T16:31:30.412 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:30 smithi133 ceph-mon[32691]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:30 smithi116 ceph-mon[32712]: Deploying daemon osd.2 on smithi133 2024-04-25T16:31:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:30 smithi116 ceph-mon[32712]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:32.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:32 smithi112 ceph-mon[29122]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:32.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:32 smithi133 ceph-mon[32691]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:32 smithi116 ceph-mon[32712]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:34.301 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:34 smithi133 ceph-mon[32691]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:34.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:34.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:34.302 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:34 smithi112 ceph-mon[29122]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:34 smithi116 ceph-mon[32712]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:36.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:36 smithi112 ceph-mon[29122]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:36 smithi133 ceph-mon[32691]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:36 smithi116 ceph-mon[32712]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:37.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.284+0000 7f8ddf7fe700 1 -- 172.21.15.133:0/3619613365 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8de8063ad0 con 0x7f8dd40afaf0 2024-04-25T16:31:37.284 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 2 on host 'smithi133' 2024-04-25T16:31:37.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8dd40afaf0 msgr2=0x7f8dd40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:37.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8dd40afaf0 0x7f8dd40b2000 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009e60 tx=0x7f8de4009c10 comp rx=0 tx=0).stop 2024-04-25T16:31:37.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 msgr2=0x7f8de81a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 0x7f8de81a1c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8de000d980 tx=0x7f8de000dd40 comp rx=0 tx=0).stop 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 shutdown_connections 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8de8109eb0 0x7f8de81a1c60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8dd40afaf0 0x7f8dd40b2000 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de806cbd0 0x7f8de819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 --2- 172.21.15.133:0/3619613365 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8de8107340 0x7f8de819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 >> 172.21.15.133:0/3619613365 conn(0x7f8de80fe740 msgr2=0x7f8de8105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 shutdown_connections 2024-04-25T16:31:37.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:31:37.286+0000 7f8df0385700 1 -- 172.21.15.133:0/3619613365 wait complete. 2024-04-25T16:31:37.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:37.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:37.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:37 smithi133 ceph-mon[32691]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:37 smithi116 ceph-mon[32712]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:37.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:37.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:37.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:37.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:37 smithi112 ceph-mon[29122]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:37.894 DEBUG:teuthology.orchestra.run.smithi133:osd.2> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T16:31:37.896 INFO:tasks.cephadm:Deploying osd.3 on smithi112 with /dev/vg_nvme/lv_3... 2024-04-25T16:31:37.896 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T16:31:38.141 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:31:38.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:38 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:38.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:38 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:38.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:38 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:38.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:38 smithi133 ceph-mon[32691]: from='osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T16:31:38.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:38 smithi133 ceph-mon[32691]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T16:31:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:38 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:38 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:38 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:38 smithi116 ceph-mon[32712]: from='osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T16:31:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:38 smithi116 ceph-mon[32712]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T16:31:38.520 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:38 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:38.520 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:38 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:38.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:38 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:31:38.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:38 smithi112 ceph-mon[29122]: from='osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T16:31:38.521 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:38 smithi112 ceph-mon[29122]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T16:31:39.424 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-mon[32691]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T16:31:39.425 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-mon[32691]: from='osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:31:39.425 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-mon[32691]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T16:31:39.425 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:39.425 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-mon[32691]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:31:39.425 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-mon[32691]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:39.425 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2[37866]: 2024-04-25T16:31:39.248+0000 7fbedf3ca700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T16:31:39.425 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2[37866]: 2024-04-25T16:31:39.252+0000 7fbedb17f700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:31:39.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:39 smithi112 ceph-mon[29122]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T16:31:39.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:39 smithi112 ceph-mon[29122]: from='osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:31:39.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:39 smithi112 ceph-mon[29122]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T16:31:39.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:39.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:39 smithi112 ceph-mon[29122]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:31:39.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:39 smithi112 ceph-mon[29122]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:39 smithi116 ceph-mon[32712]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T16:31:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:39 smithi116 ceph-mon[32712]: from='osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:31:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:39 smithi116 ceph-mon[32712]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T16:31:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:39 smithi116 ceph-mon[32712]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:31:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:39 smithi116 ceph-mon[32712]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T16:31:40.219 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T16:31:40.220 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T16:31:40.220 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T16:31:40.220 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T16:31:40.220 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116626 s, 899 MB/s 2024-04-25T16:31:40.220 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T16:31:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:40 smithi116 ceph-mon[32712]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:31:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:40 smithi116 ceph-mon[32712]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T16:31:40.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:40 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:40.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:40 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:40.518 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:40 smithi112 ceph-mon[29122]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:31:40.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:40 smithi112 ceph-mon[29122]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T16:31:40.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:40 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:40.519 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:40 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:40.532 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:40 smithi133 ceph-mon[32691]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:31:40.532 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:40 smithi133 ceph-mon[32691]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T16:31:40.532 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:40 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:40.532 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:40 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:40.860 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_3 2024-04-25T16:31:41.107 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] boot 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi133 2024-04-25T16:31:41.501 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi133 to 17088M 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:41.502 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:31:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:31:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:31:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] boot 2024-04-25T16:31:41.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi133 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi133 to 17088M 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:41.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:31:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: osd.2 [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] boot 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi133 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi133 to 17088M 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:41.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:31:41.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:41.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:41.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:31:42.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T16:31:42.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 ceph-mon[29122]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T16:31:42.491 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:31:42.491 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T16:31:42.491 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.491 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48908]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.491 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48908]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.492 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48911]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T16:31:42.492 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48911]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.492 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48911]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.492 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48911]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.561 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T16:31:42.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 ceph-mon[32691]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T16:31:42.562 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41916]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41916]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41916]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41916]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41919]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41919]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.562 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41919]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.563 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41919]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.599 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T16:31:42.599 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 ceph-mon[32712]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T16:31:42.599 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:31:42.600 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41420]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T16:31:42.600 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41420]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.600 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41420]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.600 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41420]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.600 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41423]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T16:31:42.600 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41423]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.601 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41423]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.601 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41423]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48919]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T16:31:42.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48919]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48919]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48919]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.749 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T16:31:42.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.750 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:42 smithi112 sudo[48922]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 -- 172.21.15.112:0/1735023681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec104360 msgr2=0x7f89ec10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:42.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 --2- 172.21.15.112:0/1735023681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec104360 0x7f89ec10ac10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f89e400b0a0 tx=0x7f89e400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:31:42.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 -- 172.21.15.112:0/1735023681 shutdown_connections 2024-04-25T16:31:42.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 --2- 172.21.15.112:0/1735023681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec104360 0x7f89ec10ac10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:42.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 --2- 172.21.15.112:0/1735023681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89ec102890 0x7f89ec102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 --2- 172.21.15.112:0/1735023681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89ec1039c0 0x7f89ec103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 -- 172.21.15.112:0/1735023681 >> 172.21.15.112:0/1735023681 conn(0x7f89ec0fded0 msgr2=0x7f89ec100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 -- 172.21.15.112:0/1735023681 shutdown_connections 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.749+0000 7f89f27be700 1 -- 172.21.15.112:0/1735023681 wait complete. 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 Processor -- start 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 -- start start 2024-04-25T16:31:42.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89ec102890 0x7f89ec19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 0x7f89ec19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89ec104360 0x7f89ec1a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec10fd20 con 0x7f89ec1039c0 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec10fba0 con 0x7f89ec104360 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.750+0000 7f89f27be700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f89ec10fea0 con 0x7f89ec102890 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 0x7f89ec19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:42.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 0x7f89ec19acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44192/0 (socket says 172.21.15.112:44192) 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 -- 172.21.15.112:0/1974703654 learned_addr learned my addr 172.21.15.112:0/1974703654 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89f0d5b700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89ec104360 0x7f89ec1a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89ebfff700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89ec102890 0x7f89ec19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 -- 172.21.15.112:0/1974703654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89ec102890 msgr2=0x7f89ec19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89ec102890 0x7f89ec19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 -- 172.21.15.112:0/1974703654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89ec104360 msgr2=0x7f89ec1a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:31:42.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89ec104360 0x7f89ec1a1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 -- 172.21.15.112:0/1974703654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89ec1a22d0 con 0x7f89ec1039c0 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89eb7fe700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 0x7f89ec19acb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89e000d980 tx=0x7f89e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e0016070 con 0x7f89ec1039c0 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89ec1a25c0 con 0x7f89ec1039c0 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89e0004830 con 0x7f89ec1039c0 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e00054c0 con 0x7f89ec1039c0 2024-04-25T16:31:42.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.751+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89ec1a2b50 con 0x7f89ec1039c0 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.752+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89ec051190 con 0x7f89ec1039c0 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.754+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 15) ==== 147465+0+0 (secure 0 0 0) 0x7f89e0004220 con 0x7f89ec1039c0 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.754+0000 7f89e97fa700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f89d40afaf0 0x7f89d40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.755+0000 7f89ebfff700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f89d40afaf0 0x7f89d40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.755+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f89e00dda80 con 0x7f89ec1039c0 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.755+0000 7f89ebfff700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f89d40afaf0 0x7f89d40b2000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89dc005fd0 tx=0x7f89dc00a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:31:42.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.758+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f89e001f020 con 0x7f89ec1039c0 2024-04-25T16:31:42.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41922]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41922]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41922]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41922]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41925]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41925]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41925]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:42.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:42 smithi133 sudo[41925]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:42.962+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f89ec063ad0 con 0x7f89d40afaf0 2024-04-25T16:31:43.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41426]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T16:31:43.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41426]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:43.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41426]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:43.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41426]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:43.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41429]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T16:31:43.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41429]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T16:31:43.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41429]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T16:31:43.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:42 smithi116 sudo[41429]: pam_unix(sudo:session): session closed for user root 2024-04-25T16:31:43.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:31:43.526+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f89e00a36a0 con 0x7f89ec1039c0 2024-04-25T16:31:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T16:31:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T16:31:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.528 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:31:43.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:31:43.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:31:43.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:43.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:43 smithi112 ceph-mon[29122]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:43.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T16:31:43.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T16:31:43.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.702 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:43.703 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:43 smithi133 ceph-mon[32691]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:43.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T16:31:43.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:43.761 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:43 smithi116 ceph-mon[32712]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:44.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:44 smithi112 ceph-mon[29122]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T16:31:44.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:44 smithi112 ceph-mon[29122]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T16:31:44.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:44 smithi133 ceph-mon[32691]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T16:31:44.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:44 smithi133 ceph-mon[32691]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T16:31:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:44 smithi116 ceph-mon[32712]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T16:31:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:44 smithi116 ceph-mon[32712]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T16:31:45.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:45 smithi112 ceph-mon[29122]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:45.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:45 smithi133 ceph-mon[32691]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:45.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:45 smithi116 ceph-mon[32712]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:46.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:46 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2048836903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d45175e-d218-4c74-bcec-a037fcb395f2"}]: dispatch 2024-04-25T16:31:46.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:46 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2048836903' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d45175e-d218-4c74-bcec-a037fcb395f2"}]': finished 2024-04-25T16:31:46.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:46 smithi112 ceph-mon[29122]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T16:31:46.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:31:46.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:46 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2048836903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d45175e-d218-4c74-bcec-a037fcb395f2"}]: dispatch 2024-04-25T16:31:46.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:46 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2048836903' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d45175e-d218-4c74-bcec-a037fcb395f2"}]': finished 2024-04-25T16:31:46.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:46 smithi133 ceph-mon[32691]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T16:31:46.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:31:46.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:46 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2048836903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d45175e-d218-4c74-bcec-a037fcb395f2"}]: dispatch 2024-04-25T16:31:46.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:46 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2048836903' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d45175e-d218-4c74-bcec-a037fcb395f2"}]': finished 2024-04-25T16:31:46.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:46 smithi116 ceph-mon[32712]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T16:31:46.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:31:47.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:47 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2987507472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:47.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:47 smithi112 ceph-mon[29122]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:47.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:47 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2987507472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:47.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:47 smithi133 ceph-mon[32691]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:47.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:47 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2987507472' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:31:47.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:47 smithi116 ceph-mon[32712]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:31:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:50 smithi112 ceph-mon[29122]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-04-25T16:31:50.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:50 smithi133 ceph-mon[32691]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-04-25T16:31:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:50 smithi116 ceph-mon[32712]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-04-25T16:31:52.100 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:52 smithi112 ceph-mon[29122]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-04-25T16:31:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:52 smithi133 ceph-mon[32691]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-04-25T16:31:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:52 smithi116 ceph-mon[32712]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-04-25T16:31:54.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:54 smithi112 ceph-mon[29122]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 27 KiB/s wr, 2 op/s 2024-04-25T16:31:54.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:54 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T16:31:54.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:54 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:54.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:54 smithi133 ceph-mon[32691]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 27 KiB/s wr, 2 op/s 2024-04-25T16:31:54.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:54 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T16:31:54.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:54 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:54.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:54 smithi116 ceph-mon[32712]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.0 KiB/s rd, 27 KiB/s wr, 2 op/s 2024-04-25T16:31:54.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:54 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T16:31:54.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:54 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:31:55.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:55 smithi112 ceph-mon[29122]: Deploying daemon osd.3 on smithi112 2024-04-25T16:31:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:55 smithi133 ceph-mon[32691]: Deploying daemon osd.3 on smithi112 2024-04-25T16:31:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:55 smithi116 ceph-mon[32712]: Deploying daemon osd.3 on smithi112 2024-04-25T16:31:56.191 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:56 smithi112 ceph-mon[29122]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T16:31:56.410 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:56 smithi116 ceph-mon[32712]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T16:31:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:56 smithi133 ceph-mon[32691]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T16:31:58.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:58 smithi112 ceph-mon[29122]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 909 B/s rd, 23 KiB/s wr, 2 op/s 2024-04-25T16:31:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:58 smithi133 ceph-mon[32691]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 909 B/s rd, 23 KiB/s wr, 2 op/s 2024-04-25T16:31:58.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:58 smithi116 ceph-mon[32712]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 909 B/s rd, 23 KiB/s wr, 2 op/s 2024-04-25T16:32:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:31:59 smithi116 ceph-mon[32712]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-04-25T16:32:00.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:00.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:00.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:00.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:31:59 smithi112 ceph-mon[29122]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-04-25T16:32:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:00.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:31:59 smithi133 ceph-mon[32691]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-04-25T16:32:02.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:02 smithi112 ceph-mon[29122]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:02.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:02.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:02.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:02 smithi133 ceph-mon[32691]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:02.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:02.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:02.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:02 smithi116 ceph-mon[32712]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:02.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:02.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.026 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 3 on host 'smithi112' 2024-04-25T16:32:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.026+0000 7f89e97fa700 1 -- 172.21.15.112:0/1974703654 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f89ec063ad0 con 0x7f89d40afaf0 2024-04-25T16:32:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f89d40afaf0 msgr2=0x7f89d40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f89d40afaf0 0x7f89d40b2000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f89dc005fd0 tx=0x7f89dc00a560 comp rx=0 tx=0).stop 2024-04-25T16:32:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 msgr2=0x7f89ec19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 0x7f89ec19acb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f89e000d980 tx=0x7f89e000dc90 comp rx=0 tx=0).stop 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 shutdown_connections 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f89d40afaf0 0x7f89d40b2000 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f89ec104360 0x7f89ec1a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89ec102890 0x7f89ec19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 --2- 172.21.15.112:0/1974703654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f89ec1039c0 0x7f89ec19acb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 >> 172.21.15.112:0/1974703654 conn(0x7f89ec0fded0 msgr2=0x7f89ec106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 shutdown_connections 2024-04-25T16:32:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:03.029+0000 7f89f27be700 1 -- 172.21.15.112:0/1974703654 wait complete. 2024-04-25T16:32:03.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:03 smithi112 ceph-mon[29122]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T16:32:03.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:03.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:03 smithi133 ceph-mon[32691]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T16:32:03.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:03.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:03 smithi116 ceph-mon[32712]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T16:32:03.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:03.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:03.677 DEBUG:teuthology.orchestra.run.smithi112:osd.3> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T16:32:03.679 INFO:tasks.cephadm:Deploying osd.4 on smithi116 with /dev/vg_nvme/lv_3... 2024-04-25T16:32:03.679 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T16:32:03.929 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:32:04.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-mon[29122]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:04.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-mon[29122]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T16:32:04.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-mon[29122]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T16:32:04.229 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-mon[29122]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:32:04.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:04.230 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:32:04.087+0000 7f69c1edf700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T16:32:04.230 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:32:04 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:32:04.096+0000 7f69bdc94700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:32:04.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:04 smithi116 ceph-mon[32712]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:04.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:04 smithi116 ceph-mon[32712]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T16:32:04.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:04 smithi116 ceph-mon[32712]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T16:32:04.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:04 smithi116 ceph-mon[32712]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:32:04.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:04 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:04 smithi133 ceph-mon[32691]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:04 smithi133 ceph-mon[32691]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T16:32:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:04 smithi133 ceph-mon[32691]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T16:32:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:04 smithi133 ceph-mon[32691]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:32:04.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.259 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:05 smithi112 ceph-mon[29122]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:32:05.259 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:05 smithi112 ceph-mon[29122]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T16:32:05.259 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.259 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:05 smithi116 ceph-mon[32712]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:32:05.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:05 smithi116 ceph-mon[32712]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T16:32:05.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:05 smithi133 ceph-mon[32691]: from='osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:32:05.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:05 smithi133 ceph-mon[32691]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T16:32:05.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:05.938 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T16:32:05.938 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T16:32:05.938 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-04-25T16:32:05.938 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-04-25T16:32:05.938 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113108 s, 927 MB/s 2024-04-25T16:32:05.938 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-04-25T16:32:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:06 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:32:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:06 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:32:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:06 smithi112 ceph-mon[29122]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:06.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:06 smithi112 ceph-mon[29122]: osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] boot 2024-04-25T16:32:06.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:06 smithi112 ceph-mon[29122]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T16:32:06.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:06 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:06 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:32:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:06 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:32:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:06 smithi133 ceph-mon[32691]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:06.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:06 smithi133 ceph-mon[32691]: osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] boot 2024-04-25T16:32:06.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:06 smithi133 ceph-mon[32691]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T16:32:06.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:06 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:06.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:06 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:32:06.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:06 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:32:06.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:06 smithi116 ceph-mon[32712]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T16:32:06.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:06 smithi116 ceph-mon[32712]: osd.3 [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] boot 2024-04-25T16:32:06.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:06 smithi116 ceph-mon[32712]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T16:32:06.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:06 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:32:06.571 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_3 2024-04-25T16:32:06.818 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.180 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:07 smithi116 ceph-mon[32712]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T16:32:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:07 smithi133 ceph-mon[32691]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T16:32:07.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T16:32:07.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:07.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:07.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:07.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:07.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:07.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:07 smithi112 ceph-mon[29122]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 -- 172.21.15.116:0/2762085311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 msgr2=0x7f359c10c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 --2- 172.21.15.116:0/2762085311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c10c770 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f359400b0a0 tx=0x7f359400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 -- 172.21.15.116:0/2762085311 shutdown_connections 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 --2- 172.21.15.116:0/2762085311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c10c770 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 --2- 172.21.15.116:0/2762085311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1057b0 0x7f359c105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 --2- 172.21.15.116:0/2762085311 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102200 0x7f359c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 -- 172.21.15.116:0/2762085311 >> 172.21.15.116:0/2762085311 conn(0x7f359c0fded0 msgr2=0x7f359c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 -- 172.21.15.116:0/2762085311 shutdown_connections 2024-04-25T16:32:08.351 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.351+0000 7f35a3941700 1 -- 172.21.15.116:0/2762085311 wait complete. 2024-04-25T16:32:08.352 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:08 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi112 2024-04-25T16:32:08.352 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:08 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T16:32:08.352 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:08 smithi116 ceph-mon[32712]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:08.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 Processor -- start 2024-04-25T16:32:08.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 -- start start 2024-04-25T16:32:08.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102200 0x7f359c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:08.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:08.354 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1057b0 0x7f359c1a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10fcf0 con 0x7f359c102200 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10fb70 con 0x7f359c102b50 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a3941700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f359c10fe70 con 0x7f359c1057b0 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c19acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52238/0 (socket says 172.21.15.116:52238) 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 -- 172.21.15.116:0/699869786 learned_addr learned my addr 172.21.15.116:0/699869786 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a1ede700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1057b0 0x7f359c1a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a16dd700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102200 0x7f359c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 -- 172.21.15.116:0/699869786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1057b0 msgr2=0x7f359c1a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1057b0 0x7f359c1a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 -- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102200 msgr2=0x7f359c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:08.355 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.352+0000 7f35a0edc700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102200 0x7f359c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35a0edc700 1 -- 172.21.15.116:0/699869786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c1a2280 con 0x7f359c102b50 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35a0edc700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c19acf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c9b0 tx=0x7f358c00ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c007b00 con 0x7f359c102b50 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c1a2570 con 0x7f359c102b50 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f358c00ce50 con 0x7f359c102b50 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c018660 con 0x7f359c102b50 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.353+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c1a2b80 con 0x7f359c102b50 2024-04-25T16:32:08.356 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.354+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c051190 con 0x7f359c102b50 2024-04-25T16:32:08.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.355+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f358c00f450 con 0x7f359c102b50 2024-04-25T16:32:08.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.356+0000 7f35927fc700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f35880afb80 0x7f35880b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:08.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.356+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f358c0ddc30 con 0x7f359c102b50 2024-04-25T16:32:08.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.356+0000 7f35a16dd700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f35880afb80 0x7f35880b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:08.357 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.357+0000 7f35a16dd700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f35880afb80 0x7f35880b2090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f359c06bf50 tx=0x7f3598009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:32:08.359 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.360+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f358c014030 con 0x7f359c102b50 2024-04-25T16:32:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:08 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi112 2024-04-25T16:32:08.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:08 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T16:32:08.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:08 smithi133 ceph-mon[32691]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:08.564 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:08.565+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f359c063ad0 con 0x7f35880afb80 2024-04-25T16:32:08.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:08 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi112 2024-04-25T16:32:08.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:08 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi112 to 8544M 2024-04-25T16:32:08.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:08 smithi112 ceph-mon[29122]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:09.405 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:09 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:32:09.405 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:09 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:32:09.405 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:09 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:09.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:09 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:32:09.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:09 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:32:09.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:09 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:09.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:09 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:32:09.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:09 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:32:09.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:09 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:10.167 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:10 smithi116 ceph-mon[32712]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:32:10.167 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:10 smithi116 ceph-mon[32712]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T16:32:10.168 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:10 smithi116 ceph-mon[32712]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T16:32:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:10 smithi133 ceph-mon[32691]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:32:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:10 smithi133 ceph-mon[32691]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T16:32:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:10 smithi133 ceph-mon[32691]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T16:32:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:10 smithi112 ceph-mon[29122]: from='client.24230 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:32:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:10 smithi112 ceph-mon[29122]: pgmap v104: 1 pgs: 1 activating+degraded; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T16:32:10.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:10 smithi112 ceph-mon[29122]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-04-25T16:32:12.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T16:32:12.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/962584774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]: dispatch 2024-04-25T16:32:12.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]: dispatch 2024-04-25T16:32:12.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]': finished 2024-04-25T16:32:12.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T16:32:12.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:12.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:12 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/1806534458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:32:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T16:32:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/962584774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]: dispatch 2024-04-25T16:32:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]: dispatch 2024-04-25T16:32:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]': finished 2024-04-25T16:32:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T16:32:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:12 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/1806534458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:32:12.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-04-25T16:32:12.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/962584774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]: dispatch 2024-04-25T16:32:12.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]: dispatch 2024-04-25T16:32:12.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a0f9a634-8f5a-4dbe-a8e9-8f74872cb417"}]': finished 2024-04-25T16:32:12.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: osdmap e28: 5 total, 4 up, 5 in 2024-04-25T16:32:12.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:12.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:12 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/1806534458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:32:13.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:13 smithi112 ceph-mon[29122]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T16:32:13.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:13 smithi112 ceph-mon[29122]: Cluster is now healthy 2024-04-25T16:32:13.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:13 smithi133 ceph-mon[32691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T16:32:13.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:13 smithi133 ceph-mon[32691]: Cluster is now healthy 2024-04-25T16:32:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:13 smithi116 ceph-mon[32712]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-04-25T16:32:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:13 smithi116 ceph-mon[32712]: Cluster is now healthy 2024-04-25T16:32:14.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:14 smithi133 ceph-mon[32691]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T16:32:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:14 smithi116 ceph-mon[32712]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T16:32:14.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:14 smithi112 ceph-mon[29122]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-04-25T16:32:16.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:16 smithi133 ceph-mon[32691]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:32:16.476 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:16 smithi116 ceph-mon[32712]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:32:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:16 smithi112 ceph-mon[29122]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:32:18.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:18 smithi133 ceph-mon[32691]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:18.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:18 smithi116 ceph-mon[32712]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:18.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:18 smithi112 ceph-mon[29122]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:19.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T16:32:19.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:19 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:19.499 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T16:32:19.500 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:19 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:19.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T16:32:19.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:19 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:20.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:20 smithi133 ceph-mon[32691]: Deploying daemon osd.4 on smithi116 2024-04-25T16:32:20.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:20 smithi133 ceph-mon[32691]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:20.483 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:20 smithi116 ceph-mon[32712]: Deploying daemon osd.4 on smithi116 2024-04-25T16:32:20.483 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:20 smithi116 ceph-mon[32712]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:20.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:20 smithi112 ceph-mon[29122]: Deploying daemon osd.4 on smithi116 2024-04-25T16:32:20.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:20 smithi112 ceph-mon[29122]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:22.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:22 smithi116 ceph-mon[32712]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:22 smithi133 ceph-mon[32691]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:22 smithi112 ceph-mon[29122]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:32:24.387 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:24 smithi116 ceph-mon[32712]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T16:32:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:24 smithi133 ceph-mon[32691]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T16:32:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:24 smithi112 ceph-mon[29122]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-25T16:32:25.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:25.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:25.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:25.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:25 smithi116 ceph-mon[32712]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:25.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:25.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:25.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:25.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:25 smithi112 ceph-mon[29122]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:25.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:25.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:25.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:25.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:25 smithi133 ceph-mon[32691]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:28.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:28 smithi112 ceph-mon[29122]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:28.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:28.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:28.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:28 smithi116 ceph-mon[32712]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:28.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:28.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:28.373 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.431 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 4 on host 'smithi116' 2024-04-25T16:32:28.431 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.432+0000 7f35927fc700 1 -- 172.21.15.116:0/699869786 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f359c063ad0 con 0x7f35880afb80 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f35880afb80 msgr2=0x7f35880b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f35880afb80 0x7f35880b2090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f359c06bf50 tx=0x7f3598009190 comp rx=0 tx=0).stop 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 msgr2=0x7f359c19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c19acf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c9b0 tx=0x7f358c00ccc0 comp rx=0 tx=0).stop 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 shutdown_connections 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f359c1057b0 0x7f359c1a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f359c102200 0x7f359c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f35880afb80 0x7f35880b2090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 --2- 172.21.15.116:0/699869786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c102b50 0x7f359c19acf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 >> 172.21.15.116:0/699869786 conn(0x7f359c0fded0 msgr2=0x7f359c108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 shutdown_connections 2024-04-25T16:32:28.434 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:32:28.434+0000 7f35a3941700 1 -- 172.21.15.116:0/699869786 wait complete. 2024-04-25T16:32:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:28 smithi133 ceph-mon[32691]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.136 DEBUG:teuthology.orchestra.run.smithi116:osd.4> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T16:32:29.138 INFO:tasks.cephadm:Deploying osd.5 on smithi133 with /dev/vg_nvme/lv_3... 2024-04-25T16:32:29.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T16:32:29.382 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:32:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:29 smithi133 ceph-mon[32691]: from='osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T16:32:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:29 smithi133 ceph-mon[32691]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T16:32:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:29 smithi133 ceph-mon[32691]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:29.639 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.639 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.639 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:29 smithi116 ceph-mon[32712]: from='osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T16:32:29.639 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:29 smithi116 ceph-mon[32712]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T16:32:29.639 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:29.639 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:29 smithi116 ceph-mon[32712]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:29.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:29.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:29 smithi112 ceph-mon[29122]: from='osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T16:32:29.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:29 smithi112 ceph-mon[29122]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T16:32:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:29.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:29 smithi112 ceph-mon[29122]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:30.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:30 smithi133 ceph-mon[32691]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T16:32:30.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:30 smithi133 ceph-mon[32691]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T16:32:30.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:30 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:30.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:30 smithi133 ceph-mon[32691]: from='osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:32:30.690 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:30 smithi133 ceph-mon[32691]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:32:30.695 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-mon[32712]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T16:32:30.695 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-mon[32712]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T16:32:30.695 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:30.696 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-mon[32712]: from='osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:32:30.696 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-mon[32712]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:32:30.696 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4[44259]: 2024-04-25T16:32:30.437+0000 7fa5656e1700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T16:32:30.696 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:32:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4[44259]: 2024-04-25T16:32:30.445+0000 7fa561496700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:32:30.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:30 smithi112 ceph-mon[29122]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T16:32:30.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:30 smithi112 ceph-mon[29122]: osdmap e29: 5 total, 4 up, 5 in 2024-04-25T16:32:30.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:30 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:30.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:30 smithi112 ceph-mon[29122]: from='osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:32:30.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:30 smithi112 ceph-mon[29122]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:32:31.378 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T16:32:31.378 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T16:32:31.378 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-25T16:32:31.378 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-25T16:32:31.378 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0138814 s, 755 MB/s 2024-04-25T16:32:31.378 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-25T16:32:31.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:32:31.675 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:32:31.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:32:31.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T16:32:31.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:31.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:31.676 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:31 smithi133 ceph-mon[32691]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:31.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:31 smithi116 ceph-mon[32712]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:31.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:32:31.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:32:31.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:32:31.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T16:32:31.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:31.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:31.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:31 smithi112 ceph-mon[29122]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-04-25T16:32:32.045 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2024-04-25T16:32:32.286 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:32:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] boot 2024-04-25T16:32:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi116 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:32 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.747 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] boot 2024-04-25T16:32:32.747 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi116 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:32.748 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:32 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: osd.4 [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] boot 2024-04-25T16:32:32.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: osdmap e31: 5 total, 5 up, 5 in 2024-04-25T16:32:32.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi116 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:32.760 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:32 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:33 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi116 to 8544M 2024-04-25T16:32:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:33 smithi133 ceph-mon[32691]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:33 smithi133 ceph-mon[32691]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T16:32:33.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:33 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi116 to 8544M 2024-04-25T16:32:33.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:33 smithi116 ceph-mon[32712]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:33.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:33 smithi116 ceph-mon[32712]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T16:32:33.761 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:33 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi116 to 8544M 2024-04-25T16:32:33.761 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:33 smithi112 ceph-mon[29122]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:33.761 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:33 smithi112 ceph-mon[29122]: osdmap e32: 5 total, 5 up, 5 in 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 -- 172.21.15.133:0/1234069564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f4102b40 msgr2=0x7f21f410c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 --2- 172.21.15.133:0/1234069564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f4102b40 0x7f21f410c760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b0a0 tx=0x7f21ec00b3b0 comp rx=0 tx=0).stop 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 -- 172.21.15.133:0/1234069564 shutdown_connections 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 --2- 172.21.15.133:0/1234069564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f4102b40 0x7f21f410c760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 --2- 172.21.15.133:0/1234069564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f41057a0 0x7f21f4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 --2- 172.21.15.133:0/1234069564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21f41021f0 0x7f21f4102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 -- 172.21.15.133:0/1234069564 >> 172.21.15.133:0/1234069564 conn(0x7f21f40fdee0 msgr2=0x7f21f4100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 -- 172.21.15.133:0/1234069564 shutdown_connections 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.896+0000 7f21fbef0700 1 -- 172.21.15.133:0/1234069564 wait complete. 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 Processor -- start 2024-04-25T16:32:33.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 -- start start 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 0x7f21f419a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4102b40 0x7f21f419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21f41057a0 0x7f21f41a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21f410fce0 con 0x7f21f4102b40 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f21f410fb60 con 0x7f21f41057a0 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21fbef0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f21f410fe60 con 0x7f21f41021f0 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21f9c8c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 0x7f21f419a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21f9c8c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 0x7f21f419a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58612/0 (socket says 172.21.15.133:58612) 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.897+0000 7f21f9c8c700 1 -- 172.21.15.133:0/2160569425 learned_addr learned my addr 172.21.15.133:0/2160569425 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f948b700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4102b40 0x7f21f419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21fa48d700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21f41057a0 0x7f21f41a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f9c8c700 1 -- 172.21.15.133:0/2160569425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21f41057a0 msgr2=0x7f21f41a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f9c8c700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21f41057a0 0x7f21f41a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:33.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f9c8c700 1 -- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4102b40 msgr2=0x7f21f419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f9c8c700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4102b40 0x7f21f419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f9c8c700 1 -- 172.21.15.133:0/2160569425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f41a2350 con 0x7f21f41021f0 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21f9c8c700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 0x7f21f419a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b4d0 tx=0x7f21f000b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0015020 con 0x7f21f41021f0 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f41a2640 con 0x7f21f41021f0 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21f0011ce0 con 0x7f21f41021f0 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0010c40 con 0x7f21f41021f0 2024-04-25T16:32:33.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.898+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a2c50 con 0x7f21f41021f0 2024-04-25T16:32:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.899+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f4051190 con 0x7f21f41021f0 2024-04-25T16:32:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.900+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f21f0018020 con 0x7f21f41021f0 2024-04-25T16:32:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.901+0000 7f21eaffd700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f21e00afb80 0x7f21e00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.901+0000 7f21f948b700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f21e00afb80 0x7f21e00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.901+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f21f00dd810 con 0x7f21f41021f0 2024-04-25T16:32:33.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.902+0000 7f21f948b700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f21e00afb80 0x7f21e00b2090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005950 tx=0x7f21e400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:32:33.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:33.905+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21f001f020 con 0x7f21f41021f0 2024-04-25T16:32:34.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:34.110+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f21f4063ad0 con 0x7f21e00afb80 2024-04-25T16:32:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:34 smithi133 ceph-mon[32691]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:32:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:32:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:32:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:34 smithi116 ceph-mon[32712]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:32:34.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:34.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:34 smithi112 ceph-mon[29122]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:32:34.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:32:34.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:32:34.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:35.707 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:35 smithi133 ceph-mon[32691]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:35.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:35 smithi116 ceph-mon[32712]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:35.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:35 smithi112 ceph-mon[29122]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:37.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:37 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/894099720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]: dispatch 2024-04-25T16:32:37.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:37 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]: dispatch 2024-04-25T16:32:37.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:37 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]': finished 2024-04-25T16:32:37.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:37 smithi112 ceph-mon[29122]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T16:32:37.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:37.349 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:37 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/894099720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]: dispatch 2024-04-25T16:32:37.349 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:37 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]: dispatch 2024-04-25T16:32:37.349 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:37 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]': finished 2024-04-25T16:32:37.349 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:37 smithi133 ceph-mon[32691]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T16:32:37.349 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:37 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/894099720' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]: dispatch 2024-04-25T16:32:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:37 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]: dispatch 2024-04-25T16:32:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:37 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3a17fcaf-3f2f-4fef-b666-a6c9927dc38a"}]': finished 2024-04-25T16:32:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:37 smithi116 ceph-mon[32712]: osdmap e33: 6 total, 5 up, 6 in 2024-04-25T16:32:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:38.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:38 smithi116 ceph-mon[32712]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:38.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:38 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/3946314601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:32:38.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:38 smithi112 ceph-mon[29122]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:38.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:38 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/3946314601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:32:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:38 smithi133 ceph-mon[32691]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:38 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/3946314601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:32:40.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:40 smithi112 ceph-mon[29122]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:40.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:40 smithi133 ceph-mon[32691]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:40 smithi116 ceph-mon[32712]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:42.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:42 smithi133 ceph-mon[32691]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:42.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:42 smithi112 ceph-mon[29122]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:42.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:42 smithi116 ceph-mon[32712]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:44.306 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:44 smithi133 ceph-mon[32691]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:44.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:44 smithi112 ceph-mon[29122]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:44.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:44 smithi116 ceph-mon[32712]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:45.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T16:32:45.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:45.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T16:32:45.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:45.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T16:32:45.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:46 smithi112 ceph-mon[29122]: Deploying daemon osd.5 on smithi133 2024-04-25T16:32:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:46 smithi112 ceph-mon[29122]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:46.354 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:46 smithi133 ceph-mon[32691]: Deploying daemon osd.5 on smithi133 2024-04-25T16:32:46.354 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:46 smithi133 ceph-mon[32691]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:46 smithi116 ceph-mon[32712]: Deploying daemon osd.5 on smithi133 2024-04-25T16:32:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:46 smithi116 ceph-mon[32712]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:48.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:48 smithi112 ceph-mon[29122]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:48.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:48 smithi133 ceph-mon[32691]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:48 smithi116 ceph-mon[32712]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:50.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:50 smithi133 ceph-mon[32691]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:50.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:50.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:50.453 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:50 smithi116 ceph-mon[32712]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:50 smithi112 ceph-mon[29122]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:52.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:52 smithi133 ceph-mon[32691]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:52 smithi116 ceph-mon[32712]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:52 smithi112 ceph-mon[29122]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:53.922 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:53.922 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:53.922 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:53.922 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:53.922 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:53.923 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:53 smithi133 ceph-mon[32691]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:53.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.974+0000 7f21eaffd700 1 -- 172.21.15.133:0/2160569425 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f21f4063ad0 con 0x7f21e00afb80 2024-04-25T16:32:53.975 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 5 on host 'smithi133' 2024-04-25T16:32:53.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f21e00afb80 msgr2=0x7f21e00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:53.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f21e00afb80 0x7f21e00b2090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f21e4005950 tx=0x7f21e400a300 comp rx=0 tx=0).stop 2024-04-25T16:32:53.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 msgr2=0x7f21f419a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:53.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 0x7f21f419a7f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b4d0 tx=0x7f21f000b7e0 comp rx=0 tx=0).stop 2024-04-25T16:32:53.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 shutdown_connections 2024-04-25T16:32:53.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f21e00afb80 0x7f21e00b2090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21f41057a0 0x7f21f41a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21f41021f0 0x7f21f419a7f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 --2- 172.21.15.133:0/2160569425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21f4102b40 0x7f21f419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 >> 172.21.15.133:0/2160569425 conn(0x7f21f40fdee0 msgr2=0x7f21f4108740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:32:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 shutdown_connections 2024-04-25T16:32:53.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:32:53.977+0000 7f21fbef0700 1 -- 172.21.15.133:0/2160569425 wait complete. 2024-04-25T16:32:54.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:54.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:54.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:53 smithi116 ceph-mon[32712]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:54.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:54.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:54.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:53 smithi112 ceph-mon[29122]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:54.676 DEBUG:teuthology.orchestra.run.smithi133:osd.5> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T16:32:54.679 INFO:tasks.cephadm:Deploying osd.6 on smithi112 with /dev/vg_nvme/lv_2... 2024-04-25T16:32:54.679 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T16:32:54.905 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:54 smithi133 ceph-mon[32691]: from='osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T16:32:54.905 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:54 smithi133 ceph-mon[32691]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T16:32:54.905 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:54 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:54.905 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:54 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.905 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:54 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:54.924 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:32:55.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:54 smithi116 ceph-mon[32712]: from='osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T16:32:55.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:54 smithi116 ceph-mon[32712]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T16:32:55.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:54 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:55.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:54 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:55.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:54 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:55.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:54 smithi112 ceph-mon[29122]: from='osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T16:32:55.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:54 smithi112 ceph-mon[29122]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T16:32:55.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:54 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:32:55.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:54 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:55.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:54 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:55.941 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:55 smithi133 ceph-mon[32691]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T16:32:55.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:55 smithi133 ceph-mon[32691]: from='osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:32:55.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:55 smithi133 ceph-mon[32691]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T16:32:55.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:55.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:55 smithi133 ceph-mon[32691]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:32:55.942 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:55 smithi133 ceph-mon[32691]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:56.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:55 smithi116 ceph-mon[32712]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T16:32:56.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:55 smithi116 ceph-mon[32712]: from='osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:32:56.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:55 smithi116 ceph-mon[32712]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T16:32:56.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:55 smithi116 ceph-mon[32712]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:32:56.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:55 smithi116 ceph-mon[32712]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:56.071 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:55 smithi112 ceph-mon[29122]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T16:32:56.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:55 smithi112 ceph-mon[29122]: from='osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:32:56.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:55 smithi112 ceph-mon[29122]: osdmap e34: 6 total, 5 up, 6 in 2024-04-25T16:32:56.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:55 smithi112 ceph-mon[29122]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:32:56.072 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:55 smithi112 ceph-mon[29122]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:56.200 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:32:56.119+0000 7f23b7efe700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T16:32:56.200 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:32:56.125+0000 7f23afcab700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:32:56.910 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:56 smithi116 ceph-mon[32712]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:32:56.910 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:56 smithi116 ceph-mon[32712]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T16:32:56.910 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:56 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.910 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:56 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.910 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:56 smithi116 ceph-mon[32712]: from='osd.5 ' entity='osd.5' 2024-04-25T16:32:56.931 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T16:32:56.931 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T16:32:56.931 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T16:32:56.931 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T16:32:56.931 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115589 s, 907 MB/s 2024-04-25T16:32:56.932 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:56 smithi112 ceph-mon[29122]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:56 smithi112 ceph-mon[29122]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:56 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:56 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.932 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:56 smithi112 ceph-mon[29122]: from='osd.5 ' entity='osd.5' 2024-04-25T16:32:56.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-mon[32691]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:32:56.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-mon[32691]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T16:32:56.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:56.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:56 smithi133 ceph-mon[32691]: from='osd.5 ' entity='osd.5' 2024-04-25T16:32:57.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_2 2024-04-25T16:32:57.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:32:57.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:32:57.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:57.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] boot 2024-04-25T16:32:57.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T16:32:57.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:57.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T16:32:57.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:57 smithi133 ceph-mon[32691]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:57.923 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:32:57.962 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] boot 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T16:32:57.963 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:57 smithi112 ceph-mon[29122]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: osd.5 [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] boot 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: osdmap e36: 6 total, 6 up, 6 in 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: osdmap e37: 6 total, 6 up, 6 in 2024-04-25T16:32:58.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:57 smithi116 ceph-mon[32712]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-04-25T16:32:59.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi133 2024-04-25T16:32:59.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:59.046 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:59.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi133 to 8544M 2024-04-25T16:32:59.049 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:59.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:59.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.050 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:58 smithi112 ceph-mon[29122]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T16:32:59.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi133 2024-04-25T16:32:59.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi133 to 8544M 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:58 smithi133 ceph-mon[32691]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi133 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi133 to 8544M 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:32:59.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:32:59.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:32:59.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:58 smithi116 ceph-mon[32712]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T16:32:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 -- 172.21.15.112:0/776565096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908107680 msgr2=0x7f0908109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 --2- 172.21.15.112:0/776565096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908107680 0x7f0908109aa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009990 tx=0x7f08f8009ca0 comp rx=0 tx=0).stop 2024-04-25T16:32:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 -- 172.21.15.112:0/776565096 shutdown_connections 2024-04-25T16:32:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 --2- 172.21.15.112:0/776565096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 0x7f090810c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 --2- 172.21.15.112:0/776565096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908104d20 0x7f0908107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:59.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 --2- 172.21.15.112:0/776565096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908107680 0x7f0908109aa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 -- 172.21.15.112:0/776565096 >> 172.21.15.112:0/776565096 conn(0x7f09080fe760 msgr2=0x7f0908100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 -- 172.21.15.112:0/776565096 shutdown_connections 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.594+0000 7f091063b700 1 -- 172.21.15.112:0/776565096 wait complete. 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 Processor -- start 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 -- start start 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 0x7f090819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908107680 0x7f090819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 0x7f09081a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f090810fcf0 con 0x7f0908104d20 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f090810fb70 con 0x7f0908109fe0 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f091063b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f090810fe70 con 0x7f0908107680 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 0x7f090819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 0x7f090819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38424/0 (socket says 172.21.15.112:38424) 2024-04-25T16:32:59.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 -- 172.21.15.112:0/2795338540 learned_addr learned my addr 172.21.15.112:0/2795338540 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090dbd6700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908107680 0x7f090819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 -- 172.21.15.112:0/2795338540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908107680 msgr2=0x7f090819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908107680 0x7f090819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 -- 172.21.15.112:0/2795338540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 msgr2=0x7f09081a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090ebd8700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 0x7f09081a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 0x7f09081a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 -- 172.21.15.112:0/2795338540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09081a2370 con 0x7f0908104d20 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090dbd6700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908107680 0x7f090819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090ebd8700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 0x7f09081a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.595+0000 7f090e3d7700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 0x7f090819a810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f090400d980 tx=0x7f090400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.596+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904017070 con 0x7f0908104d20 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.596+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09081a2660 con 0x7f0908104d20 2024-04-25T16:32:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.596+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0904010460 con 0x7f0908104d20 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.596+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090400de60 con 0x7f0908104d20 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.596+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09081a2c70 con 0x7f0908104d20 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.597+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0908051190 con 0x7f0908104d20 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.598+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f0904010610 con 0x7f0908104d20 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.599+0000 7f08ff7fe700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f08f40afb80 0x7f08f40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.599+0000 7f090dbd6700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f08f40afb80 0x7f08f40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.599+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f09040dd910 con 0x7f0908104d20 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.599+0000 7f090dbd6700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f08f40afb80 0x7f08f40b2090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08f8005b40 tx=0x7f08f8005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:32:59.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.603+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09040a73b0 con 0x7f0908104d20 2024-04-25T16:32:59.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:32:59.811+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0908063ad0 con 0x7f08f40afb80 2024-04-25T16:33:00.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:59 smithi112 ceph-mon[29122]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:00.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:00.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:00.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:32:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:00.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:59 smithi133 ceph-mon[32691]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:00.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:00.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:00.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:32:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:59 smithi116 ceph-mon[32712]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:32:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:01.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:00 smithi112 ceph-mon[29122]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:01.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:00 smithi133 ceph-mon[32691]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:01.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:00 smithi116 ceph-mon[32712]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:02.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:01 smithi112 ceph-mon[29122]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:02.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:01 smithi133 ceph-mon[32691]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:02.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:01 smithi116 ceph-mon[32712]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:03.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:02 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2395312709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29"}]: dispatch 2024-04-25T16:33:03.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:02 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2395312709' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29"}]': finished 2024-04-25T16:33:03.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:02 smithi112 ceph-mon[29122]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T16:33:03.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:03.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:02 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2395312709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29"}]: dispatch 2024-04-25T16:33:03.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:02 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2395312709' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29"}]': finished 2024-04-25T16:33:03.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:02 smithi133 ceph-mon[32691]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T16:33:03.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:03.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:02 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2395312709' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29"}]: dispatch 2024-04-25T16:33:03.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:02 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2395312709' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29"}]': finished 2024-04-25T16:33:03.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:02 smithi116 ceph-mon[32712]: osdmap e39: 7 total, 6 up, 7 in 2024-04-25T16:33:03.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:04.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:03 smithi133 ceph-mon[32691]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T16:33:04.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:03 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3248570061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:04.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:03 smithi116 ceph-mon[32712]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T16:33:04.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:03 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3248570061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:04.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:03 smithi112 ceph-mon[29122]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T16:33:04.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:03 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3248570061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:06 smithi112 ceph-mon[29122]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:33:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:06 smithi133 ceph-mon[32691]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:33:06.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:06 smithi116 ceph-mon[32712]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:33:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:08 smithi112 ceph-mon[29122]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T16:33:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:08 smithi133 ceph-mon[32691]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T16:33:08.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:08 smithi116 ceph-mon[32712]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T16:33:10.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:10 smithi112 ceph-mon[29122]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:33:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:10 smithi133 ceph-mon[32691]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:33:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:10 smithi116 ceph-mon[32712]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:33:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:11 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T16:33:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:11 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:11.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:11 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T16:33:11.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:11 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:11.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:11 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T16:33:11.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:11 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:12 smithi112 ceph-mon[29122]: Deploying daemon osd.6 on smithi112 2024-04-25T16:33:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:12 smithi112 ceph-mon[29122]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:33:12.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:12 smithi133 ceph-mon[32691]: Deploying daemon osd.6 on smithi112 2024-04-25T16:33:12.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:12 smithi133 ceph-mon[32691]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:33:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:12 smithi116 ceph-mon[32712]: Deploying daemon osd.6 on smithi112 2024-04-25T16:33:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:12 smithi116 ceph-mon[32712]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:33:14.438 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:14 smithi112 ceph-mon[29122]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T16:33:14.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:14 smithi133 ceph-mon[32691]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T16:33:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:14 smithi116 ceph-mon[32712]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-25T16:33:16.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:16 smithi133 ceph-mon[32691]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:16 smithi116 ceph-mon[32712]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:16 smithi112 ceph-mon[29122]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:17 smithi133 ceph-mon[32691]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:17.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:17.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:17.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:17.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:17 smithi116 ceph-mon[32712]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:17.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:17.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:17.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:17.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:17 smithi112 ceph-mon[29122]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:19.772 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 6 on host 'smithi112' 2024-04-25T16:33:19.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.773+0000 7f08ff7fe700 1 -- 172.21.15.112:0/2795338540 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0908063ad0 con 0x7f08f40afb80 2024-04-25T16:33:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f08f40afb80 msgr2=0x7f08f40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:19.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f08f40afb80 0x7f08f40b2090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f08f8005b40 tx=0x7f08f8005a90 comp rx=0 tx=0).stop 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 msgr2=0x7f090819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 0x7f090819a810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f090400d980 tx=0x7f090400dd40 comp rx=0 tx=0).stop 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 shutdown_connections 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f08f40afb80 0x7f08f40b2090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0908109fe0 0x7f09081a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0908104d20 0x7f090819a810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 --2- 172.21.15.112:0/2795338540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0908107680 0x7f090819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 >> 172.21.15.112:0/2795338540 conn(0x7f09080fe760 msgr2=0x7f0908105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 shutdown_connections 2024-04-25T16:33:19.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:33:19.775+0000 7f091063b700 1 -- 172.21.15.112:0/2795338540 wait complete. 2024-04-25T16:33:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:20 smithi112 ceph-mon[29122]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:20 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:20.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:20 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:20.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:20 smithi133 ceph-mon[32691]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:20.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:20 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:20.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:20 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:20.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:20 smithi116 ceph-mon[32712]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:20.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:20 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:20.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:20 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:20.811 DEBUG:teuthology.orchestra.run.smithi112:osd.6> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T16:33:20.814 INFO:tasks.cephadm:Deploying osd.7 on smithi116 with /dev/vg_nvme/lv_2... 2024-04-25T16:33:20.814 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T16:33:21.061 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:33:21.184 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:21 smithi116 ceph-mon[32712]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T16:33:21.184 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:21.184 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:21.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:21 smithi112 ceph-mon[29122]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T16:33:21.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:21.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:21.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:21 smithi133 ceph-mon[32691]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T16:33:21.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:21.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-mon[29122]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-mon[29122]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T16:33:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-mon[29122]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T16:33:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-mon[29122]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:33:22.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:22.413 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:22 smithi116 ceph-mon[32712]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:22.414 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:22 smithi116 ceph-mon[32712]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T16:33:22.414 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:22 smithi116 ceph-mon[32712]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T16:33:22.414 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:22 smithi116 ceph-mon[32712]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:33:22.414 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:22 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:22 smithi133 ceph-mon[32691]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:22 smithi133 ceph-mon[32691]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T16:33:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:22 smithi133 ceph-mon[32691]: osdmap e40: 7 total, 6 up, 7 in 2024-04-25T16:33:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:22 smithi133 ceph-mon[32691]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:33:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:22 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:22.818 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:33:22.540+0000 7f6ba5031700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T16:33:22.818 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:33:22 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:33:22.548+0000 7f6b9cdde700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:33:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T16:33:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T16:33:23.251 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-04-25T16:33:23.251 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-04-25T16:33:23.251 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128098 s, 819 MB/s 2024-04-25T16:33:23.251 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-04-25T16:33:23.276 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:23 smithi116 ceph-mon[32712]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:33:23.277 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:23 smithi116 ceph-mon[32712]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T16:33:23.277 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:23.277 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:23.277 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:23 smithi116 ceph-mon[32712]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' 2024-04-25T16:33:23.363 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:23 smithi112 ceph-mon[29122]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:33:23.363 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:23 smithi112 ceph-mon[29122]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T16:33:23.363 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:23.363 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:23.364 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:23 smithi112 ceph-mon[29122]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' 2024-04-25T16:33:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:23 smithi133 ceph-mon[32691]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:33:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:23 smithi133 ceph-mon[32691]: osdmap e41: 7 total, 6 up, 7 in 2024-04-25T16:33:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:23 smithi133 ceph-mon[32691]: from='osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708]' entity='osd.6' 2024-04-25T16:33:23.948 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_2 2024-04-25T16:33:24.192 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] boot 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T16:33:24.236 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] boot 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T16:33:24.343 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:33:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:33:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-04-25T16:33:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: osd.6 [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] boot 2024-04-25T16:33:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: osdmap e42: 7 total, 7 up, 7 in 2024-04-25T16:33:24.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:33:25.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 -- 172.21.15.116:0/4043032908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f181057b0 msgr2=0x7f9f18105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:25.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/4043032908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f181057b0 0x7f9f18105bc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009990 tx=0x7f9f0c009ca0 comp rx=0 tx=0).stop 2024-04-25T16:33:25.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 -- 172.21.15.116:0/4043032908 shutdown_connections 2024-04-25T16:33:25.919 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/4043032908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f18102b50 0x7f9f1810c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:25.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/4043032908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f181057b0 0x7f9f18105bc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:25.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/4043032908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f18102200 0x7f9f18102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:25.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 -- 172.21.15.116:0/4043032908 >> 172.21.15.116:0/4043032908 conn(0x7f9f180fded0 msgr2=0x7f9f18100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:33:25.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 -- 172.21.15.116:0/4043032908 shutdown_connections 2024-04-25T16:33:25.920 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.920+0000 7f9f1c85b700 1 -- 172.21.15.116:0/4043032908 wait complete. 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 Processor -- start 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 -- start start 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f18102200 0x7f9f1819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 0x7f9f1819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f181057b0 0x7f9f181a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810fca0 con 0x7f9f18102200 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810fb20 con 0x7f9f18102b50 2024-04-25T16:33:25.921 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1c85b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1810fe20 con 0x7f9f181057b0 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 0x7f9f1819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f1659c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f18102200 0x7f9f1819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 0x7f9f1819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42092/0 (socket says 172.21.15.116:42092) 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 -- 172.21.15.116:0/3075794150 learned_addr learned my addr 172.21.15.116:0/3075794150 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f16d9d700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f181057b0 0x7f9f181a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 -- 172.21.15.116:0/3075794150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f181057b0 msgr2=0x7f9f181a1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f181057b0 0x7f9f181a1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 -- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f18102200 msgr2=0x7f9f1819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f18102200 0x7f9f1819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.921+0000 7f9f15d9b700 1 -- 172.21.15.116:0/3075794150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f181a2350 con 0x7f9f18102b50 2024-04-25T16:33:25.922 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.922+0000 7f9f15d9b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 0x7f9f1819ad30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800c920 tx=0x7f9f0800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:33:25.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.924+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004a30 con 0x7f9f18102b50 2024-04-25T16:33:25.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.924+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f080054d0 con 0x7f9f18102b50 2024-04-25T16:33:25.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.924+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f181a2640 con 0x7f9f18102b50 2024-04-25T16:33:25.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.924+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08007c70 con 0x7f9f18102b50 2024-04-25T16:33:25.923 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.924+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f181a2b80 con 0x7f9f18102b50 2024-04-25T16:33:25.924 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.925+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f18051190 con 0x7f9f18102b50 2024-04-25T16:33:25.925 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.926+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f9f08005020 con 0x7f9f18102b50 2024-04-25T16:33:25.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.927+0000 7f9f077fe700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f9f000afb80 0x7f9f000b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:25.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.927+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7f9f080ddda0 con 0x7f9f18102b50 2024-04-25T16:33:25.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.927+0000 7f9f1659c700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f9f000afb80 0x7f9f000b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:25.926 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.928+0000 7f9f1659c700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f9f000afb80 0x7f9f000b2090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b6c0 tx=0x7f9f0c00b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:33:25.931 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:25.932+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9f08014030 con 0x7f9f18102b50 2024-04-25T16:33:25.931 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi112 2024-04-25T16:33:25.931 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:25.931 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:25.931 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:25.931 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:25.932 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:25 smithi116 ceph-mon[32712]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T16:33:26.054 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi112 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:26.055 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:26.056 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.056 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:26.056 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:25 smithi133 ceph-mon[32691]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T16:33:26.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi112 2024-04-25T16:33:26.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi112 to 5696M 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:26.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:25 smithi112 ceph-mon[29122]: osdmap e43: 7 total, 7 up, 7 in 2024-04-25T16:33:26.140 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:26.140+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9f18063ad0 con 0x7f9f000afb80 2024-04-25T16:33:27.001 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:26 smithi116 ceph-mon[32712]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:27.002 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:27.002 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:27.002 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:27.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:26 smithi133 ceph-mon[32691]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:27.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:27.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:27.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:27.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:26 smithi112 ceph-mon[29122]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:27.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:27.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:27.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:27.993 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:27 smithi116 ceph-mon[32712]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:28.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:27 smithi133 ceph-mon[32691]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:28.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:27 smithi112 ceph-mon[29122]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:29.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:28 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/2185709254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]: dispatch 2024-04-25T16:33:29.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:28 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]: dispatch 2024-04-25T16:33:29.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:28 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]': finished 2024-04-25T16:33:29.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:28 smithi133 ceph-mon[32691]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T16:33:29.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:28 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:29.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:28 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/2185709254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]: dispatch 2024-04-25T16:33:29.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:28 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]: dispatch 2024-04-25T16:33:29.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:28 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]': finished 2024-04-25T16:33:29.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:28 smithi116 ceph-mon[32712]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T16:33:29.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:28 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:28 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/2185709254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]: dispatch 2024-04-25T16:33:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:28 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]: dispatch 2024-04-25T16:33:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:28 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c1b323-2808-4b81-9164-36836a690a83"}]': finished 2024-04-25T16:33:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:28 smithi112 ceph-mon[29122]: osdmap e44: 8 total, 7 up, 8 in 2024-04-25T16:33:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:28 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:29 smithi133 ceph-mon[32691]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/1701826156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:29 smithi116 ceph-mon[32712]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/1701826156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:29 smithi112 ceph-mon[29122]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/1701826156' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:32.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:32 smithi112 ceph-mon[29122]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:32.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:32 smithi133 ceph-mon[32691]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:32 smithi116 ceph-mon[32712]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:34.209 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:34 smithi116 ceph-mon[32712]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:34 smithi112 ceph-mon[29122]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:34.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:34 smithi133 ceph-mon[32691]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:36.306 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:36 smithi116 ceph-mon[32712]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:36.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:36 smithi112 ceph-mon[29122]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:36 smithi133 ceph-mon[32691]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:37.215 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T16:33:37.215 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:37.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T16:33:37.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T16:33:37.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:38.193 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:38 smithi116 ceph-mon[32712]: Deploying daemon osd.7 on smithi116 2024-04-25T16:33:38.193 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:38 smithi116 ceph-mon[32712]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:38 smithi133 ceph-mon[32691]: Deploying daemon osd.7 on smithi116 2024-04-25T16:33:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:38 smithi133 ceph-mon[32691]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:38.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:38 smithi112 ceph-mon[29122]: Deploying daemon osd.7 on smithi116 2024-04-25T16:33:38.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:38 smithi112 ceph-mon[29122]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:40.206 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:40 smithi116 ceph-mon[32712]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:40.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:40 smithi133 ceph-mon[32691]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:40.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:40 smithi112 ceph-mon[29122]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:42.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:42 smithi133 ceph-mon[32691]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:42.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:42 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:42.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:42 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:42.490 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:42 smithi116 ceph-mon[32712]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:42.490 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:42 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:42.493 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:42 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:42 smithi112 ceph-mon[29122]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:42 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:42 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:43.417 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:43.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:43.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:44.416 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:44 smithi116 ceph-mon[32712]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:44.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:44 smithi133 ceph-mon[32691]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:44.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:44 smithi112 ceph-mon[29122]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:46.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.056+0000 7f9f077fe700 1 -- 172.21.15.116:0/3075794150 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9f18063ad0 con 0x7f9f000afb80 2024-04-25T16:33:46.056 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 7 on host 'smithi116' 2024-04-25T16:33:46.057 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f9f000afb80 msgr2=0x7f9f000b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f9f000afb80 0x7f9f000b2090 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00b6c0 tx=0x7f9f0c00b720 comp rx=0 tx=0).stop 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 msgr2=0x7f9f1819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 0x7f9f1819ad30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800c920 tx=0x7f9f0800cc30 comp rx=0 tx=0).stop 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 shutdown_connections 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f181057b0 0x7f9f181a1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f18102200 0x7f9f1819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f9f000afb80 0x7f9f000b2090 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 --2- 172.21.15.116:0/3075794150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f18102b50 0x7f9f1819ad30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:46.058 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.058+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 >> 172.21.15.116:0/3075794150 conn(0x7f9f180fded0 msgr2=0x7f9f18108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:33:46.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.059+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 shutdown_connections 2024-04-25T16:33:46.059 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:33:46.059+0000 7f9f1c85b700 1 -- 172.21.15.116:0/3075794150 wait complete. 2024-04-25T16:33:46.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:46.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:46.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:46.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T16:33:46.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T16:33:46.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T16:33:46.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:46.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:46.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T16:33:46.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T16:33:46.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:46.747 DEBUG:teuthology.orchestra.run.smithi116:osd.7> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T16:33:46.749 INFO:tasks.cephadm:Deploying osd.8 on smithi133 with /dev/vg_nvme/lv_2... 2024-04-25T16:33:46.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T16:33:46.996 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: from='osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:47.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-mon[32712]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:33:47.259 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:33:47.133+0000 7efc94082700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T16:33:47.260 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:33:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:33:47.141+0000 7efc8fe37700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:33:47.340 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:47.340 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:47.340 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T16:33:47.341 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T16:33:47.341 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: from='osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:33:47.341 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:47.341 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:47 smithi133 ceph-mon[32691]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:33:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:33:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T16:33:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: osdmap e45: 8 total, 7 up, 8 in 2024-04-25T16:33:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: from='osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:33:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:47 smithi112 ceph-mon[29122]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:33:48.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:48 smithi133 ceph-mon[32691]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:48.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:48 smithi133 ceph-mon[32691]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:33:48.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:48 smithi133 ceph-mon[32691]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T16:33:48.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:48 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:48.356 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:48 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:48 smithi116 ceph-mon[32712]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:48 smithi116 ceph-mon[32712]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:33:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:48 smithi116 ceph-mon[32712]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T16:33:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:48 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:48 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:48.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:48 smithi112 ceph-mon[29122]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-04-25T16:33:48.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:48 smithi112 ceph-mon[29122]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:33:48.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:48 smithi112 ceph-mon[29122]: osdmap e46: 8 total, 7 up, 8 in 2024-04-25T16:33:48.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:48 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:48.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:48 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:48.938 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T16:33:48.938 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T16:33:48.938 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-25T16:33:48.938 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-25T16:33:48.938 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011469 s, 914 MB/s 2024-04-25T16:33:48.938 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-25T16:33:49.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:49 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:33:49.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:49 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:33:49.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:49 smithi116 ceph-mon[32712]: osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] boot 2024-04-25T16:33:49.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:49 smithi116 ceph-mon[32712]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T16:33:49.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:49.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:49 smithi116 ceph-mon[32712]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T16:33:49.342 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:49 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:33:49.342 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:49 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:33:49.342 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:49 smithi133 ceph-mon[32691]: osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] boot 2024-04-25T16:33:49.342 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:49 smithi133 ceph-mon[32691]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T16:33:49.343 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:49.343 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:49 smithi133 ceph-mon[32691]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T16:33:49.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:49 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:33:49.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:49 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:33:49.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:49 smithi112 ceph-mon[29122]: osd.7 [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] boot 2024-04-25T16:33:49.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:49 smithi112 ceph-mon[29122]: osdmap e47: 8 total, 8 up, 8 in 2024-04-25T16:33:49.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:33:49.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:49 smithi112 ceph-mon[29122]: osdmap e48: 8 total, 8 up, 8 in 2024-04-25T16:33:49.670 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2024-04-25T16:33:49.916 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:33:50.288 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:50.289 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:50 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:50.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:50.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:50 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:33:50.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:50 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:33:51.401 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:51 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi116 2024-04-25T16:33:51.402 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:51 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi116 to 5696M 2024-04-25T16:33:51.402 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:51 smithi133 ceph-mon[32691]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T16:33:51.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:51 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi116 2024-04-25T16:33:51.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:51 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi116 to 5696M 2024-04-25T16:33:51.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:51 smithi116 ceph-mon[32712]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T16:33:51.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:51 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi116 2024-04-25T16:33:51.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:51 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi116 to 5696M 2024-04-25T16:33:51.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:51 smithi112 ceph-mon[29122]: osdmap e49: 8 total, 8 up, 8 in 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 -- 172.21.15.133:0/3067742988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 msgr2=0x7f4148103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 --2- 172.21.15.133:0/3067742988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148103740 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4134009990 tx=0x7f4134009ca0 comp rx=0 tx=0).stop 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 -- 172.21.15.133:0/3067742988 shutdown_connections 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 --2- 172.21.15.133:0/3067742988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4148103c80 0x7f414810b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 --2- 172.21.15.133:0/3067742988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4148104620 0x7f4148104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 --2- 172.21.15.133:0/3067742988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148103740 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 -- 172.21.15.133:0/3067742988 >> 172.21.15.133:0/3067742988 conn(0x7f41480fdef0 msgr2=0x7f41481003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 -- 172.21.15.133:0/3067742988 shutdown_connections 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.652+0000 7f414eb37700 1 -- 172.21.15.133:0/3067742988 wait complete. 2024-04-25T16:33:51.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 Processor -- start 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 -- start start 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4148103c80 0x7f4148105680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4148104620 0x7f4148105bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f414810ec40 con 0x7f4148104620 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f414810eac0 con 0x7f4148103c80 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414eb37700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f414810edc0 con 0x7f4148103330 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59458/0 (socket says 172.21.15.133:59458) 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 -- 172.21.15.133:0/3885810832 learned_addr learned my addr 172.21.15.133:0/3885810832 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:33:51.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414d0d4700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4148104620 0x7f4148105bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f4147fff700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4148103c80 0x7f4148105680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 -- 172.21.15.133:0/3885810832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4148103c80 msgr2=0x7f4148105680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4148103c80 0x7f4148105680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 -- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4148104620 msgr2=0x7f4148105bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4148104620 0x7f4148105bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.653+0000 7f414c8d3700 1 -- 172.21.15.133:0/3885810832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4148106320 con 0x7f4148103330 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.654+0000 7f414c8d3700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148106fd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b500 tx=0x7f413c00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.654+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c015020 con 0x7f4148103330 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.654+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f413c011ce0 con 0x7f4148103330 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.654+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41481a4940 con 0x7f4148103330 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.654+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413c010d40 con 0x7f4148103330 2024-04-25T16:33:51.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.654+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41481a4ef0 con 0x7f4148103330 2024-04-25T16:33:51.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.655+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4148051190 con 0x7f4148103330 2024-04-25T16:33:51.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.657+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f413c018020 con 0x7f4148103330 2024-04-25T16:33:51.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.657+0000 7f4145ffb700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f41380afc00 0x7f41380b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:33:51.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.657+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f413c0dddd0 con 0x7f4148103330 2024-04-25T16:33:51.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.658+0000 7f4147fff700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f41380afc00 0x7f41380b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:33:51.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.658+0000 7f4147fff700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f41380afc00 0x7f41380b2110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41340038a0 tx=0x7f413401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:33:51.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.661+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f413c01f020 con 0x7f4148103330 2024-04-25T16:33:51.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:33:51.865+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f41481a5120 con 0x7f41380afc00 2024-04-25T16:33:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:52 smithi133 ceph-mon[32691]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:52 smithi116 ceph-mon[32712]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:52 smithi112 ceph-mon[29122]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:33:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:33:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:33:53.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:53 smithi133 ceph-mon[32691]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:53.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:53 smithi116 ceph-mon[32712]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:53.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:53 smithi112 ceph-mon[29122]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:33:54.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:54 smithi133 ceph-mon[32691]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:54.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:54 smithi116 ceph-mon[32712]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:54.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:54 smithi112 ceph-mon[29122]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:33:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:55 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/112040209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]: dispatch 2024-04-25T16:33:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:55 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]: dispatch 2024-04-25T16:33:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:55 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]': finished 2024-04-25T16:33:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:55 smithi133 ceph-mon[32691]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T16:33:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:33:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:55 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/112040209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]: dispatch 2024-04-25T16:33:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:55 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]: dispatch 2024-04-25T16:33:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:55 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]': finished 2024-04-25T16:33:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:55 smithi116 ceph-mon[32712]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T16:33:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:33:55.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:55 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/112040209' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]: dispatch 2024-04-25T16:33:55.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:55 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]: dispatch 2024-04-25T16:33:55.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:55 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a"}]': finished 2024-04-25T16:33:55.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:55 smithi112 ceph-mon[29122]: osdmap e50: 9 total, 8 up, 9 in 2024-04-25T16:33:55.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:33:56.425 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:56 smithi116 ceph-mon[32712]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-04-25T16:33:56.425 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:56 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/3209030574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:56 smithi133 ceph-mon[32691]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-04-25T16:33:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:56 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/3209030574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:56.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:56 smithi112 ceph-mon[29122]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-04-25T16:33:56.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:56 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/3209030574' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:33:58.444 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:33:58 smithi133 ceph-mon[32691]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-04-25T16:33:58.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:33:58 smithi116 ceph-mon[32712]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-04-25T16:33:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:33:58 smithi112 ceph-mon[29122]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-04-25T16:34:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:00 smithi133 ceph-mon[32691]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-25T16:34:00.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:00 smithi116 ceph-mon[32712]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-25T16:34:00.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:00 smithi112 ceph-mon[29122]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-25T16:34:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:02 smithi133 ceph-mon[32691]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T16:34:02.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:02 smithi116 ceph-mon[32712]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T16:34:02.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:02 smithi112 ceph-mon[29122]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T16:34:03.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T16:34:03.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:03.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T16:34:03.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:03.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T16:34:03.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:04 smithi133 ceph-mon[32691]: Deploying daemon osd.8 on smithi133 2024-04-25T16:34:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:04 smithi133 ceph-mon[32691]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T16:34:04.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:04 smithi116 ceph-mon[32712]: Deploying daemon osd.8 on smithi133 2024-04-25T16:34:04.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:04 smithi116 ceph-mon[32712]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T16:34:04.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:04 smithi112 ceph-mon[29122]: Deploying daemon osd.8 on smithi133 2024-04-25T16:34:04.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:04 smithi112 ceph-mon[29122]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-25T16:34:06.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:06 smithi133 ceph-mon[32691]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:06.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:06 smithi116 ceph-mon[32712]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:06.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:06 smithi112 ceph-mon[29122]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:08 smithi133 ceph-mon[32691]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:08 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:08 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:08 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:08.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:08 smithi116 ceph-mon[32712]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:08.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:08.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:08.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:08.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:08 smithi112 ceph-mon[29122]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:08.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:08 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:08.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:08 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:08.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:08 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:10 smithi133 ceph-mon[32691]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:10 smithi116 ceph-mon[32712]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:10 smithi112 ceph-mon[29122]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:12.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.030+0000 7f4145ffb700 1 -- 172.21.15.133:0/3885810832 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f41481a5120 con 0x7f41380afc00 2024-04-25T16:34:12.030 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 8 on host 'smithi133' 2024-04-25T16:34:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f41380afc00 msgr2=0x7f41380b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f41380afc00 0x7f41380b2110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f41340038a0 tx=0x7f413401a040 comp rx=0 tx=0).stop 2024-04-25T16:34:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 msgr2=0x7f4148106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148106fd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f413c00b500 tx=0x7f413c00b8c0 comp rx=0 tx=0).stop 2024-04-25T16:34:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 shutdown_connections 2024-04-25T16:34:12.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4148104620 0x7f4148105bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:12.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4148103330 0x7f4148106fd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:12.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f41380afc00 0x7f41380b2110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:12.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 --2- 172.21.15.133:0/3885810832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4148103c80 0x7f4148105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:12.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 >> 172.21.15.133:0/3885810832 conn(0x7f41480fdef0 msgr2=0x7f41480ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:34:12.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 shutdown_connections 2024-04-25T16:34:12.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:34:12.033+0000 7f414eb37700 1 -- 172.21.15.133:0/3885810832 wait complete. 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T16:34:12.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.351 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:12.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:12.711 DEBUG:teuthology.orchestra.run.smithi133:osd.8> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T16:34:12.713 INFO:tasks.cephadm:Deploying osd.9 on smithi112 with /dev/vg_nvme/lv_1... 2024-04-25T16:34:12.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T16:34:12.961 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:34:13.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:13 smithi112 ceph-mon[29122]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T16:34:13.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:13 smithi112 ceph-mon[29122]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T16:34:13.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:13 smithi112 ceph-mon[29122]: from='osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:34:13.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:13.335 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:13 smithi112 ceph-mon[29122]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:34:13.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-mon[32691]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T16:34:13.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-mon[32691]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T16:34:13.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-mon[32691]: from='osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:34:13.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:13.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-mon[32691]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:34:13.451 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:34:13.212+0000 7f40045ac700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T16:34:13.452 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:34:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:34:13.222+0000 7f4000361700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:34:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:13 smithi116 ceph-mon[32712]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T16:34:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:13 smithi116 ceph-mon[32712]: osdmap e51: 9 total, 8 up, 9 in 2024-04-25T16:34:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:13 smithi116 ceph-mon[32712]: from='osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:34:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:13 smithi116 ceph-mon[32712]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:34:14.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:14 smithi133 ceph-mon[32691]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:14.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:14 smithi133 ceph-mon[32691]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:34:14.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:14 smithi133 ceph-mon[32691]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T16:34:14.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:14 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:14.273 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:14 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:14.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:14 smithi116 ceph-mon[32712]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:14 smithi116 ceph-mon[32712]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:34:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:14 smithi116 ceph-mon[32712]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T16:34:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:14 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:14 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:14.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:14 smithi112 ceph-mon[29122]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:14.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:14 smithi112 ceph-mon[29122]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:34:14.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:14 smithi112 ceph-mon[29122]: osdmap e52: 9 total, 8 up, 9 in 2024-04-25T16:34:14.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:14 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:14.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:14 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 10+0 records in 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr:10+0 records out 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115923 s, 905 MB/s 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr: stderr: 2024-04-25T16:34:15.123 INFO:teuthology.orchestra.run.smithi112.stderr:--> Zapping successful for: 2024-04-25T16:34:15.407 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:15 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:34:15.407 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:15 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:34:15.407 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:15 smithi133 ceph-mon[32691]: osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] boot 2024-04-25T16:34:15.408 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:15 smithi133 ceph-mon[32691]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T16:34:15.408 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:15.435 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:15 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:34:15.435 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:15 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:34:15.435 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:15 smithi112 ceph-mon[29122]: osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] boot 2024-04-25T16:34:15.435 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:15 smithi112 ceph-mon[29122]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T16:34:15.436 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:15.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:15 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:34:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:15 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:34:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:15 smithi116 ceph-mon[32712]: osd.8 [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] boot 2024-04-25T16:34:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:15 smithi116 ceph-mon[32712]: osdmap e53: 9 total, 9 up, 9 in 2024-04-25T16:34:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:34:15.843 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi112:vg_nvme/lv_1 2024-04-25T16:34:16.081 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.513 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:16.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:16.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 223 MiB used, 715 GiB / 715 GiB avail 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: osdmap e54: 9 total, 9 up, 9 in 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:16.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:16.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:16.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:16.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:17.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:17 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi133 2024-04-25T16:34:17.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:17 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi133 to 5696M 2024-04-25T16:34:17.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:17 smithi116 ceph-mon[32712]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T16:34:17.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:17 smithi116 ceph-mon[32712]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:17.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:17 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi133 2024-04-25T16:34:17.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:17 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi133 to 5696M 2024-04-25T16:34:17.559 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:17 smithi112 ceph-mon[29122]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T16:34:17.560 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:17 smithi112 ceph-mon[29122]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:17 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi133 2024-04-25T16:34:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:17 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi133 to 5696M 2024-04-25T16:34:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:17 smithi133 ceph-mon[32691]: osdmap e55: 9 total, 9 up, 9 in 2024-04-25T16:34:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:17 smithi133 ceph-mon[32691]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 -- 172.21.15.112:0/1544805460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104260 msgr2=0x7f8ee4104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 --2- 172.21.15.112:0/1544805460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104260 0x7f8ee4104670 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009990 tx=0x7f8ed8009ca0 comp rx=0 tx=0).stop 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 -- 172.21.15.112:0/1544805460 shutdown_connections 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 --2- 172.21.15.112:0/1544805460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104c00 0x7f8ee410b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 --2- 172.21.15.112:0/1544805460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee4103130 0x7f8ee4103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 --2- 172.21.15.112:0/1544805460 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104260 0x7f8ee4104670 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 -- 172.21.15.112:0/1544805460 >> 172.21.15.112:0/1544805460 conn(0x7f8ee40fe790 msgr2=0x7f8ee4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 -- 172.21.15.112:0/1544805460 shutdown_connections 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 -- 172.21.15.112:0/1544805460 wait complete. 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 Processor -- start 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.987+0000 7f8eea986700 1 -- start start 2024-04-25T16:34:17.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8eea986700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee4103130 0x7f8ee4076320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8eea986700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104260 0x7f8ee41a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8eea986700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 0x7f8ee41a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8eea986700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee410eba0 con 0x7f8ee4104c00 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8eea986700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee410ea20 con 0x7f8ee4104260 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8eea986700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee410ed20 con 0x7f8ee4103130 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 0x7f8ee41a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 0x7f8ee41a3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56010/0 (socket says 172.21.15.112:56010) 2024-04-25T16:34:17.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 -- 172.21.15.112:0/1666218159 learned_addr learned my addr 172.21.15.112:0/1666218159 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee37fe700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104260 0x7f8ee41a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee3fff700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee4103130 0x7f8ee4076320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 -- 172.21.15.112:0/1666218159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee4103130 msgr2=0x7f8ee4076320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee4103130 0x7f8ee4076320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 -- 172.21.15.112:0/1666218159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104260 msgr2=0x7f8ee41a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104260 0x7f8ee41a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee8f23700 1 -- 172.21.15.112:0/1666218159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee41a4590 con 0x7f8ee4104c00 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.988+0000 7f8ee37fe700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104260 0x7f8ee41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.989+0000 7f8ee8f23700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 0x7f8ee41a3f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8edc007ee0 tx=0x7f8edc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.989+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc004bb0 con 0x7f8ee4104c00 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.989+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8edc004d50 con 0x7f8ee4104c00 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.989+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8edc005920 con 0x7f8ee4104c00 2024-04-25T16:34:17.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.989+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee41a4880 con 0x7f8ee4104c00 2024-04-25T16:34:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.989+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee41a4d70 con 0x7f8ee4104c00 2024-04-25T16:34:17.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.990+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee4051190 con 0x7f8ee4104c00 2024-04-25T16:34:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.991+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f8edc01a020 con 0x7f8ee4104c00 2024-04-25T16:34:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.992+0000 7f8ee17fa700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ecc0afb80 0x7f8ecc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.992+0000 7f8ee3fff700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ecc0afb80 0x7f8ecc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.992+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f8edc0ddfa0 con 0x7f8ee4104c00 2024-04-25T16:34:17.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.992+0000 7f8ee3fff700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ecc0afb80 0x7f8ecc0b2090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4104160 tx=0x7f8ed4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:34:17.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:17.996+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8edc010070 con 0x7f8ee4104c00 2024-04-25T16:34:18.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:18.202+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8ee4063ad0 con 0x7f8ecc0afb80 2024-04-25T16:34:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:18 smithi116 ceph-mon[32712]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:34:18.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:18 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:34:18.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:18 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:34:18.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:18 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:18.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:18 smithi112 ceph-mon[29122]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:34:18.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:34:18.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:34:18.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:18 smithi133 ceph-mon[32691]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi112:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:34:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:18 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:34:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:18 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:34:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:18 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:19.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:19 smithi116 ceph-mon[32712]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T16:34:19.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:19 smithi112 ceph-mon[29122]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T16:34:19.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:19 smithi133 ceph-mon[32691]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-04-25T16:34:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:22 smithi133 ceph-mon[32691]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T16:34:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:22 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/908202436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29c6af5-9461-4eef-b445-bcb8a7bfed1e"}]: dispatch 2024-04-25T16:34:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:22 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/908202436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a29c6af5-9461-4eef-b445-bcb8a7bfed1e"}]': finished 2024-04-25T16:34:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:22 smithi133 ceph-mon[32691]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T16:34:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:22 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:22.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:22 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/119164682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:34:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:22 smithi116 ceph-mon[32712]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T16:34:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:22 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/908202436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29c6af5-9461-4eef-b445-bcb8a7bfed1e"}]: dispatch 2024-04-25T16:34:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:22 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/908202436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a29c6af5-9461-4eef-b445-bcb8a7bfed1e"}]': finished 2024-04-25T16:34:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:22 smithi116 ceph-mon[32712]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T16:34:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:22 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:22.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:22 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/119164682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:34:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:22 smithi112 ceph-mon[29122]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-25T16:34:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/908202436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a29c6af5-9461-4eef-b445-bcb8a7bfed1e"}]: dispatch 2024-04-25T16:34:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/908202436' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a29c6af5-9461-4eef-b445-bcb8a7bfed1e"}]': finished 2024-04-25T16:34:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:22 smithi112 ceph-mon[29122]: osdmap e56: 10 total, 9 up, 10 in 2024-04-25T16:34:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:22 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:22.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/119164682' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:34:24.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:24 smithi133 ceph-mon[32691]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T16:34:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:24 smithi116 ceph-mon[32712]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T16:34:24.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:24 smithi112 ceph-mon[29122]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-25T16:34:26.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:26 smithi133 ceph-mon[32691]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T16:34:26.468 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:26 smithi112 ceph-mon[29122]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T16:34:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:26 smithi116 ceph-mon[32712]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-25T16:34:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:28 smithi133 ceph-mon[32691]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:34:28.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:28 smithi116 ceph-mon[32712]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:34:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:28 smithi112 ceph-mon[29122]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:34:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T16:34:29.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:29 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T16:34:29.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:29 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:29.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T16:34:29.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:29 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:30 smithi112 ceph-mon[29122]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:30 smithi112 ceph-mon[29122]: Deploying daemon osd.9 on smithi112 2024-04-25T16:34:30.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:30 smithi133 ceph-mon[32691]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:30.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:30 smithi133 ceph-mon[32691]: Deploying daemon osd.9 on smithi112 2024-04-25T16:34:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:30 smithi116 ceph-mon[32712]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:30 smithi116 ceph-mon[32712]: Deploying daemon osd.9 on smithi112 2024-04-25T16:34:32.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:32 smithi112 ceph-mon[29122]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:32.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:32 smithi133 ceph-mon[32691]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:32 smithi116 ceph-mon[32712]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:34.242 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:34 smithi112 ceph-mon[29122]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:34.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:34 smithi133 ceph-mon[32691]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:34 smithi116 ceph-mon[32712]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:36.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:36 smithi112 ceph-mon[29122]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:36.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:36 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:36.227 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:36 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:36.228 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:36 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:36 smithi133 ceph-mon[32691]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:36 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:36 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:36 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:36 smithi116 ceph-mon[32712]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:36 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:36 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:36 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:38.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:38 smithi112 ceph-mon[29122]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:38 smithi133 ceph-mon[32691]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:38.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:38 smithi116 ceph-mon[32712]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:39.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.326+0000 7f8ee17fa700 1 -- 172.21.15.112:0/1666218159 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8ee4063ad0 con 0x7f8ecc0afb80 2024-04-25T16:34:39.325 INFO:teuthology.orchestra.run.smithi112.stdout:Created osd(s) 9 on host 'smithi112' 2024-04-25T16:34:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.328+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ecc0afb80 msgr2=0x7f8ecc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.328+0000 7f8eea986700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ecc0afb80 0x7f8ecc0b2090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4104160 tx=0x7f8ed4009260 comp rx=0 tx=0).stop 2024-04-25T16:34:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.328+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 msgr2=0x7f8ee41a3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:39.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.328+0000 7f8eea986700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 0x7f8ee41a3f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8edc007ee0 tx=0x7f8edc00d3b0 comp rx=0 tx=0).stop 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.328+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 shutdown_connections 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ee4104c00 0x7f8ee41a3f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ee4103130 0x7f8ee4076320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ecc0afb80 0x7f8ecc0b2090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 --2- 172.21.15.112:0/1666218159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4104260 0x7f8ee41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 >> 172.21.15.112:0/1666218159 conn(0x7f8ee40fe790 msgr2=0x7f8ee4107490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 shutdown_connections 2024-04-25T16:34:39.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:34:39.329+0000 7f8eea986700 1 -- 172.21.15.112:0/1666218159 wait complete. 2024-04-25T16:34:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:39 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:39 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:39.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:39 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:40.161 DEBUG:teuthology.orchestra.run.smithi112:osd.9> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T16:34:40.163 INFO:tasks.cephadm:Deploying osd.10 on smithi116 with /dev/vg_nvme/lv_1... 2024-04-25T16:34:40.163 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T16:34:40.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-mon[29122]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T16:34:40.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-mon[29122]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T16:34:40.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-mon[29122]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:34:40.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:40.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:40.412 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:34:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:40 smithi116 ceph-mon[32712]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T16:34:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:40 smithi116 ceph-mon[32712]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T16:34:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:40 smithi116 ceph-mon[32712]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:34:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:40 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:40.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:40 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:40.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:40 smithi133 ceph-mon[32691]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T16:34:40.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:40 smithi133 ceph-mon[32691]: osdmap e57: 10 total, 9 up, 10 in 2024-04-25T16:34:40.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:40 smithi133 ceph-mon[32691]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]: dispatch 2024-04-25T16:34:40.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:40 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:40.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:40 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:34:41.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:34:40.753+0000 7f99cc244700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T16:34:41.066 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:34:40 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:34:40.764+0000 7f99c37f0700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:34:41.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:41 smithi112 ceph-mon[29122]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:34:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:41 smithi112 ceph-mon[29122]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T16:34:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:41 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:41 smithi112 ceph-mon[29122]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' 2024-04-25T16:34:41.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:41 smithi112 ceph-mon[29122]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:41.570 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:41 smithi116 ceph-mon[32712]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:34:41.571 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:41 smithi116 ceph-mon[32712]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T16:34:41.571 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:41.571 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:41 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:41.571 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:41 smithi116 ceph-mon[32712]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' 2024-04-25T16:34:41.571 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:41 smithi116 ceph-mon[32712]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:41 smithi133 ceph-mon[32691]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi112", "root=default"]}]': finished 2024-04-25T16:34:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:41 smithi133 ceph-mon[32691]: osdmap e58: 10 total, 9 up, 10 in 2024-04-25T16:34:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:41 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:41 smithi133 ceph-mon[32691]: from='osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642]' entity='osd.9' 2024-04-25T16:34:41.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:41 smithi133 ceph-mon[32691]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 251 MiB used, 804 GiB / 805 GiB avail 2024-04-25T16:34:42.399 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:42 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:34:42.400 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:42 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:34:42.400 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:42 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:42.400 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:42 smithi116 ceph-mon[32712]: osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] boot 2024-04-25T16:34:42.400 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:42 smithi116 ceph-mon[32712]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T16:34:42.400 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:42 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T16:34:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T16:34:42.587 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10+0 records in 2024-04-25T16:34:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:10+0 records out 2024-04-25T16:34:42.587 INFO:teuthology.orchestra.run.smithi116.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120372 s, 871 MB/s 2024-04-25T16:34:42.587 INFO:teuthology.orchestra.run.smithi116.stderr:--> Zapping successful for: 2024-04-25T16:34:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:42 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:34:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:42 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:34:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:42 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:42 smithi112 ceph-mon[29122]: osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] boot 2024-04-25T16:34:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:42 smithi112 ceph-mon[29122]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T16:34:42.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:42 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:42.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:42 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:34:42.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:42 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:34:42.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:42 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:42.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:42 smithi133 ceph-mon[32691]: osd.9 [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] boot 2024-04-25T16:34:42.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:42 smithi133 ceph-mon[32691]: osdmap e59: 10 total, 10 up, 10 in 2024-04-25T16:34:42.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:42 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:34:43.362 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi116:vg_nvme/lv_1 2024-04-25T16:34:43.599 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:43.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:43.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:43 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.973 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:43.974 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:43.974 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:43.974 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:43.974 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:43 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: osdmap e60: 10 total, 10 up, 10 in 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:44.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:34:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:34:44.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:43 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:34:45.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:44 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi112 2024-04-25T16:34:45.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:44 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T16:34:45.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:44 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi112 2024-04-25T16:34:45.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:44 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T16:34:45.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:44 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi112 2024-04-25T16:34:45.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:44 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi112 to 4272M 2024-04-25T16:34:45.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 -- 172.21.15.116:0/968102843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 msgr2=0x7efdfc105bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:45.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 --2- 172.21.15.116:0/968102843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc105bc0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b0a0 tx=0x7efdf400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:34:45.273 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 -- 172.21.15.116:0/968102843 shutdown_connections 2024-04-25T16:34:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 --2- 172.21.15.116:0/968102843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc102b50 0x7efdfc10c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 --2- 172.21.15.116:0/968102843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc105bc0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 --2- 172.21.15.116:0/968102843 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102200 0x7efdfc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 -- 172.21.15.116:0/968102843 >> 172.21.15.116:0/968102843 conn(0x7efdfc0fded0 msgr2=0x7efdfc100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:34:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 -- 172.21.15.116:0/968102843 shutdown_connections 2024-04-25T16:34:45.274 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.274+0000 7efe00992700 1 -- 172.21.15.116:0/968102843 wait complete. 2024-04-25T16:34:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 Processor -- start 2024-04-25T16:34:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 -- start start 2024-04-25T16:34:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc102200 0x7efdfc19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102b50 0x7efdfc19ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:45.276 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc1a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc10fc60 con 0x7efdfc102b50 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.275+0000 7efe00992700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc10fae0 con 0x7efdfc1057b0 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efe00992700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc10fde0 con 0x7efdfc102200 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc1a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc1a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44120/0 (socket says 172.21.15.116:44120) 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 -- 172.21.15.116:0/100694327 learned_addr learned my addr 172.21.15.116:0/100694327 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfa59c700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc102200 0x7efdfc19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdf9d9b700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102b50 0x7efdfc19ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 -- 172.21.15.116:0/100694327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc102200 msgr2=0x7efdfc19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc102200 0x7efdfc19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 -- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102b50 msgr2=0x7efdfc19ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102b50 0x7efdfc19ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 -- 172.21.15.116:0/100694327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc1a2300 con 0x7efdfc1057b0 2024-04-25T16:34:45.277 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdfad9d700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc1a1c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e970 tx=0x7efdec00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:34:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec00c960 con 0x7efdfc1057b0 2024-04-25T16:34:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc1a2590 con 0x7efdfc1057b0 2024-04-25T16:34:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec010430 con 0x7efdfc1057b0 2024-04-25T16:34:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec018680 con 0x7efdfc1057b0 2024-04-25T16:34:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.276+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc1a2b20 con 0x7efdfc1057b0 2024-04-25T16:34:45.278 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.277+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc051190 con 0x7efdfc1057b0 2024-04-25T16:34:45.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.279+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7efdec0105e0 con 0x7efdfc1057b0 2024-04-25T16:34:45.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.280+0000 7efdf37fe700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efde80afb80 0x7efde80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:34:45.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.280+0000 7efdfa59c700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efde80afb80 0x7efde80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:34:45.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.280+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7efdec0deb30 con 0x7efdfc1057b0 2024-04-25T16:34:45.279 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.280+0000 7efdfa59c700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efde80afb80 0x7efde80b2090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b070 tx=0x7efdf40060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:34:45.282 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.284+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdec0a7e70 con 0x7efdfc1057b0 2024-04-25T16:34:45.486 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:34:45.487+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7efdfc063ad0 con 0x7efde80afb80 2024-04-25T16:34:46.094 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:45 smithi116 ceph-mon[32712]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:46.094 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:34:46.094 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:34:46.094 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:46.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:45 smithi112 ceph-mon[29122]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:46.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:34:46.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:34:46.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:46.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:45 smithi133 ceph-mon[32691]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:46.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:34:46.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:34:46.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:47.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:46 smithi112 ceph-mon[29122]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:34:47.118 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:46 smithi116 ceph-mon[32712]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:34:47.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:46 smithi133 ceph-mon[32691]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi116:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:34:48.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:47 smithi112 ceph-mon[29122]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:48.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:47 smithi133 ceph-mon[32691]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:48.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:47 smithi116 ceph-mon[32712]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:49.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:48 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/331492350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]: dispatch 2024-04-25T16:34:49.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:48 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]: dispatch 2024-04-25T16:34:49.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:48 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]': finished 2024-04-25T16:34:49.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:48 smithi116 ceph-mon[32712]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T16:34:49.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:48 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:34:49.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:48 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/331492350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]: dispatch 2024-04-25T16:34:49.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:48 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]: dispatch 2024-04-25T16:34:49.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:48 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]': finished 2024-04-25T16:34:49.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:48 smithi112 ceph-mon[29122]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T16:34:49.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:48 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:34:49.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:48 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/331492350' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]: dispatch 2024-04-25T16:34:49.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:48 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]: dispatch 2024-04-25T16:34:49.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:48 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b0d3168a-8529-49d8-bd4c-1c1bd7336f60"}]': finished 2024-04-25T16:34:49.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:48 smithi133 ceph-mon[32691]: osdmap e61: 11 total, 10 up, 11 in 2024-04-25T16:34:49.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:48 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:34:50.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:49 smithi112 ceph-mon[29122]: from='client.? 172.21.15.116:0/3323466595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:34:50.099 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:49 smithi112 ceph-mon[29122]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:50.203 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:49 smithi133 ceph-mon[32691]: from='client.? 172.21.15.116:0/3323466595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:34:50.203 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:49 smithi133 ceph-mon[32691]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:50.265 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:49 smithi116 ceph-mon[32712]: from='client.? 172.21.15.116:0/3323466595' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:34:50.265 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:49 smithi116 ceph-mon[32712]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:52.428 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:52 smithi116 ceph-mon[32712]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:52 smithi133 ceph-mon[32691]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:52 smithi112 ceph-mon[29122]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:54.408 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:54 smithi116 ceph-mon[32712]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:54.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:54 smithi133 ceph-mon[32691]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:54.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:54 smithi112 ceph-mon[29122]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:56 smithi133 ceph-mon[32691]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:56 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T16:34:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:56 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:56.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:56 smithi116 ceph-mon[32712]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:56.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:56 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T16:34:56.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:56 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:56.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:56 smithi112 ceph-mon[29122]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:56.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:56 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T16:34:56.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:56 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:34:57.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:57 smithi133 ceph-mon[32691]: Deploying daemon osd.10 on smithi116 2024-04-25T16:34:57.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:57 smithi116 ceph-mon[32712]: Deploying daemon osd.10 on smithi116 2024-04-25T16:34:57.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:57 smithi112 ceph-mon[29122]: Deploying daemon osd.10 on smithi116 2024-04-25T16:34:58.121 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:34:58 smithi116 ceph-mon[32712]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:34:58 smithi133 ceph-mon[32691]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:34:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:34:58 smithi112 ceph-mon[29122]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:00.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:00 smithi116 ceph-mon[32712]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:00 smithi133 ceph-mon[32691]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:00.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:00 smithi112 ceph-mon[29122]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:02.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:02 smithi112 ceph-mon[29122]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:02.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:02.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:02.398 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:02 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:02 smithi133 ceph-mon[32691]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:02.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:02 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:02.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:02 smithi116 ceph-mon[32712]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:02.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:02 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:04.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:04 smithi116 ceph-mon[32712]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:04.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:04 smithi133 ceph-mon[32691]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:04.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:04 smithi112 ceph-mon[29122]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T16:35:06.052 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:05 smithi116 ceph-mon[32712]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T16:35:06.053 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.053+0000 7efdf37fe700 1 -- 172.21.15.116:0/100694327 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7efdfc063ad0 con 0x7efde80afb80 2024-04-25T16:35:06.053 INFO:teuthology.orchestra.run.smithi116.stdout:Created osd(s) 10 on host 'smithi116' 2024-04-25T16:35:06.054 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efde80afb80 msgr2=0x7efde80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efde80afb80 0x7efde80b2090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdf400b070 tx=0x7efdf40060c0 comp rx=0 tx=0).stop 2024-04-25T16:35:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 msgr2=0x7efdfc1a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc1a1c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efdec00e970 tx=0x7efdec00ec80 comp rx=0 tx=0).stop 2024-04-25T16:35:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 shutdown_connections 2024-04-25T16:35:06.055 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efde80afb80 0x7efde80b2090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdfc1057b0 0x7efdfc1a1c80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdfc102200 0x7efdfc19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 --2- 172.21.15.116:0/100694327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efdfc102b50 0x7efdfc19ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 >> 172.21.15.116:0/100694327 conn(0x7efdfc0fded0 msgr2=0x7efdfc108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 shutdown_connections 2024-04-25T16:35:06.056 INFO:teuthology.orchestra.run.smithi116.stderr:2024-04-25T16:35:06.055+0000 7efe00992700 1 -- 172.21.15.116:0/100694327 wait complete. 2024-04-25T16:35:06.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:06.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:06.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:06.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T16:35:06.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:05 smithi133 ceph-mon[32691]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T16:35:06.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T16:35:06.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:05 smithi112 ceph-mon[29122]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T16:35:06.879 DEBUG:teuthology.orchestra.run.smithi116:osd.10> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T16:35:06.881 INFO:tasks.cephadm:Deploying osd.11 on smithi133 with /dev/vg_nvme/lv_1... 2024-04-25T16:35:06.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T16:35:07.115 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:07.116 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:07.116 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:35:06 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:35:06.918+0000 7f17f849d700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:35:07.132 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:07.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:06 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: osdmap e62: 11 total, 10 up, 11 in 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]: dispatch 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:06 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:08.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:07 smithi133 ceph-mon[32691]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:35:08.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:07 smithi133 ceph-mon[32691]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T16:35:08.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:08.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:08.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:07 smithi133 ceph-mon[32691]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:08.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:07 smithi116 ceph-mon[32712]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:35:08.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:07 smithi116 ceph-mon[32712]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T16:35:08.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:08.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:08.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:07 smithi116 ceph-mon[32712]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:08.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:07 smithi112 ceph-mon[29122]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi116", "root=default"]}]': finished 2024-04-25T16:35:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:07 smithi112 ceph-mon[29122]: osdmap e63: 11 total, 10 up, 11 in 2024-04-25T16:35:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:08.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:07 smithi112 ceph-mon[29122]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-04-25T16:35:09.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:08 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:35:09.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:08 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:35:09.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:08 smithi133 ceph-mon[32691]: osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] boot 2024-04-25T16:35:09.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:08 smithi133 ceph-mon[32691]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T16:35:09.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:08 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:09.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:08 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:35:09.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:08 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:35:09.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:08 smithi116 ceph-mon[32712]: osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] boot 2024-04-25T16:35:09.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:08 smithi116 ceph-mon[32712]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T16:35:09.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:08 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T16:35:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T16:35:09.336 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2024-04-25T16:35:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2024-04-25T16:35:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126471 s, 829 MB/s 2024-04-25T16:35:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2024-04-25T16:35:09.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:08 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:35:09.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:08 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:35:09.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:08 smithi112 ceph-mon[29122]: osd.10 [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] boot 2024-04-25T16:35:09.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:08 smithi112 ceph-mon[29122]: osdmap e64: 11 total, 11 up, 11 in 2024-04-25T16:35:09.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:08 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:35:10.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:09 smithi116 ceph-mon[32712]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T16:35:10.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:09 smithi116 ceph-mon[32712]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 309 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:10.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2024-04-25T16:35:10.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:09 smithi133 ceph-mon[32691]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T16:35:10.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:09 smithi133 ceph-mon[32691]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 309 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:10.306 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:35:10.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:09 smithi112 ceph-mon[29122]: osdmap e65: 11 total, 11 up, 11 in 2024-04-25T16:35:10.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:09 smithi112 ceph-mon[29122]: pgmap v232: 1 pgs: 1 remapped+peering; 577 KiB data, 309 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:10.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T16:35:10.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi116 2024-04-25T16:35:10.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:10.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:10.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:10.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi116 to 4272M 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:10.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi116 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi116 to 4272M 2024-04-25T16:35:11.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:11.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:11.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: osdmap e66: 11 total, 11 up, 11 in 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi116 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:11.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi116 to 4272M 2024-04-25T16:35:11.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:11.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:11.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:11.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:12.008 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:11 smithi133 ceph-mon[32691]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 -- 172.21.15.133:0/631094621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358104cd0 msgr2=0x7f13581070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 --2- 172.21.15.133:0/631094621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358104cd0 0x7f13581070f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1348009990 tx=0x7f1348009ca0 comp rx=0 tx=0).stop 2024-04-25T16:35:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 -- 172.21.15.133:0/631094621 shutdown_connections 2024-04-25T16:35:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 --2- 172.21.15.133:0/631094621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1358109f90 0x7f135810c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 --2- 172.21.15.133:0/631094621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358104cd0 0x7f13581070f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 --2- 172.21.15.133:0/631094621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1358107630 0x7f1358109a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 -- 172.21.15.133:0/631094621 >> 172.21.15.133:0/631094621 conn(0x7f13580fe730 msgr2=0x7f1358100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 -- 172.21.15.133:0/631094621 shutdown_connections 2024-04-25T16:35:12.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 -- 172.21.15.133:0/631094621 wait complete. 2024-04-25T16:35:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 Processor -- start 2024-04-25T16:35:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.125+0000 7f135e769700 1 -- start start 2024-04-25T16:35:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135e769700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1358104cd0 0x7f135819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135e769700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1358107630 0x7f135819acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135e769700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 0x7f13581a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135e769700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f135810fc10 con 0x7f1358107630 2024-04-25T16:35:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135e769700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f135810fa90 con 0x7f1358104cd0 2024-04-25T16:35:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135e769700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f135810fd90 con 0x7f1358109f90 2024-04-25T16:35:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 0x7f13581a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 0x7f13581a1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37544/0 (socket says 172.21.15.133:37544) 2024-04-25T16:35:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 -- 172.21.15.133:0/1365390477 learned_addr learned my addr 172.21.15.133:0/1365390477 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:35:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f1357fff700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1358104cd0 0x7f135819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f13577fe700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1358107630 0x7f135819acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 -- 172.21.15.133:0/1365390477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1358104cd0 msgr2=0x7f135819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1358104cd0 0x7f135819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 -- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1358107630 msgr2=0x7f135819acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1358107630 0x7f135819acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 -- 172.21.15.133:0/1365390477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13581a22f0 con 0x7f1358109f90 2024-04-25T16:35:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.126+0000 7f135cd06700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 0x7f13581a1c70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f135000ada0 tx=0x7f135000c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.127+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350017020 con 0x7f1358109f90 2024-04-25T16:35:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.127+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13581a25e0 con 0x7f1358109f90 2024-04-25T16:35:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.127+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f135000ce30 con 0x7f1358109f90 2024-04-25T16:35:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.127+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350005350 con 0x7f1358109f90 2024-04-25T16:35:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.127+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13581a2ad0 con 0x7f1358109f90 2024-04-25T16:35:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.128+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1358051190 con 0x7f1358109f90 2024-04-25T16:35:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.129+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f135000c980 con 0x7f1358109f90 2024-04-25T16:35:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.130+0000 7f13557fa700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f13400afb80 0x7f13400b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.130+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f13500de2f0 con 0x7f1358109f90 2024-04-25T16:35:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.130+0000 7f1357fff700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f13400afb80 0x7f13400b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.130+0000 7f1357fff700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f13400afb80 0x7f13400b2090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f134800b5c0 tx=0x7f134801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:12.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.133+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1350021020 con 0x7f1358109f90 2024-04-25T16:35:12.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:11 smithi116 ceph-mon[32712]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:12.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:12.339+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1358063ad0 con 0x7f13400afb80 2024-04-25T16:35:12.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:11 smithi112 ceph-mon[29122]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:13.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:35:13.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:35:13.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:13.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:35:13.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:35:13.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T16:35:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T16:35:13.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:14.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:13 smithi133 ceph-mon[32691]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:35:14.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:13 smithi133 ceph-mon[32691]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:14.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:13 smithi116 ceph-mon[32712]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:35:14.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:13 smithi116 ceph-mon[32712]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:14.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:13 smithi112 ceph-mon[29122]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:35:14.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:13 smithi112 ceph-mon[29122]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:16.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T16:35:16.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/2861273175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]: dispatch 2024-04-25T16:35:16.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]: dispatch 2024-04-25T16:35:16.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]': finished 2024-04-25T16:35:16.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T16:35:16.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:16.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:16 smithi133 ceph-mon[32691]: from='client.? 172.21.15.133:0/3216767549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/2861273175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]: dispatch 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]: dispatch 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]': finished 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:16 smithi116 ceph-mon[32712]: from='client.? 172.21.15.133:0/3216767549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/2861273175' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]: dispatch 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]: dispatch 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1"}]': finished 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: osdmap e67: 12 total, 11 up, 12 in 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:16.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:16 smithi112 ceph-mon[29122]: from='client.? 172.21.15.133:0/3216767549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T16:35:18.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:18 smithi133 ceph-mon[32691]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:35:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:18 smithi116 ceph-mon[32712]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:35:18.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:18 smithi112 ceph-mon[29122]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T16:35:20.405 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:20 smithi133 ceph-mon[32691]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T16:35:20.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:20 smithi116 ceph-mon[32712]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T16:35:20.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:20 smithi112 ceph-mon[29122]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T16:35:22.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:22 smithi133 ceph-mon[32691]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:35:22.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:22 smithi116 ceph-mon[32712]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:35:22.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:22 smithi112 ceph-mon[29122]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:35:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T16:35:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T16:35:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:23.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T16:35:23.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:24.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:24 smithi133 ceph-mon[32691]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:35:24.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:24 smithi133 ceph-mon[32691]: Deploying daemon osd.11 on smithi133 2024-04-25T16:35:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:24 smithi116 ceph-mon[32712]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:35:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:24 smithi116 ceph-mon[32712]: Deploying daemon osd.11 on smithi133 2024-04-25T16:35:24.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:24 smithi112 ceph-mon[29122]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T16:35:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:24 smithi112 ceph-mon[29122]: Deploying daemon osd.11 on smithi133 2024-04-25T16:35:26.394 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:26 smithi133 ceph-mon[32691]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:26 smithi116 ceph-mon[32712]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:26.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:26 smithi112 ceph-mon[29122]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:28.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:28 smithi133 ceph-mon[32691]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:28.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:28 smithi116 ceph-mon[32712]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:28 smithi112 ceph-mon[29122]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:30 smithi112 ceph-mon[29122]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:30 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:30 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:30 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:30.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:30 smithi133 ceph-mon[32691]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:30.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:30 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:30.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:30 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:30.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:30 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:30 smithi116 ceph-mon[32712]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:30 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:30 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:30 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:32.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:32 smithi112 ceph-mon[29122]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:32.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:32 smithi133 ceph-mon[32691]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:32 smithi116 ceph-mon[32712]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:33.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.058+0000 7f13557fa700 1 -- 172.21.15.133:0/1365390477 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1358063ad0 con 0x7f13400afb80 2024-04-25T16:35:33.058 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 11 on host 'smithi133' 2024-04-25T16:35:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.060+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f13400afb80 msgr2=0x7f13400b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.060+0000 7f135e769700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f13400afb80 0x7f13400b2090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f134800b5c0 tx=0x7f134801a040 comp rx=0 tx=0).stop 2024-04-25T16:35:33.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 msgr2=0x7f13581a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 0x7f13581a1c70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f135000ada0 tx=0x7f135000c330 comp rx=0 tx=0).stop 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 shutdown_connections 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1358109f90 0x7f13581a1c70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f13400afb80 0x7f13400b2090 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1358104cd0 0x7f135819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 --2- 172.21.15.133:0/1365390477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1358107630 0x7f135819acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 >> 172.21.15.133:0/1365390477 conn(0x7f13580fe730 msgr2=0x7f13581057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 shutdown_connections 2024-04-25T16:35:33.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:33.061+0000 7f135e769700 1 -- 172.21.15.133:0/1365390477 wait complete. 2024-04-25T16:35:33.378 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.378 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.378 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:33.378 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:33.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T16:35:33.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.379 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:33 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:33 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:33.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:33.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:33 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:33.739 DEBUG:teuthology.orchestra.run.smithi133:osd.11> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T16:35:33.741 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T16:35:33.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T16:35:33.978 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:35:34.236 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-mon[32691]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:34.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-mon[32691]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T16:35:34.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-mon[32691]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T16:35:34.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-mon[32691]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:35:34.237 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:34.237 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:35:34.138+0000 7fe1bd4ae700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T16:35:34.237 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:35:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:35:34.153+0000 7fe1b9263700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T16:35:34.381 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:34 smithi112 ceph-mon[29122]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:34.381 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:34 smithi112 ceph-mon[29122]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T16:35:34.381 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:34 smithi112 ceph-mon[29122]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T16:35:34.382 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:34 smithi112 ceph-mon[29122]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:35:34.382 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:34 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:34 smithi116 ceph-mon[32712]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:34 smithi116 ceph-mon[32712]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T16:35:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:34 smithi116 ceph-mon[32712]: osdmap e68: 12 total, 11 up, 12 in 2024-04-25T16:35:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:34 smithi116 ceph-mon[32712]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2024-04-25T16:35:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:34 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.272 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:35 smithi133 ceph-mon[32691]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:35:35.272 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:35 smithi133 ceph-mon[32691]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T16:35:35.272 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:35 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.272 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:35 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:35 smithi116 ceph-mon[32712]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:35:35.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:35 smithi116 ceph-mon[32712]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T16:35:35.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:35 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:35 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.548 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:35 smithi112 ceph-mon[29122]: from='osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2024-04-25T16:35:35.548 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:35 smithi112 ceph-mon[29122]: osdmap e69: 12 total, 11 up, 12 in 2024-04-25T16:35:35.548 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:35 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.548 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:35 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/1566309071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 msgr2=0x7f4cc01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/1566309071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc01049f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40099c0 tx=0x7f4cb4009cd0 comp rx=0 tx=0).stop 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/1566309071 shutdown_connections 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/1566309071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cc0104f30 0x7f4cc010b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/1566309071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 0x7f4cc0103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/1566309071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc01049f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/1566309071 >> 172.21.15.112:0/1566309071 conn(0x7f4cc00fded0 msgr2=0x7f4cc0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/1566309071 shutdown_connections 2024-04-25T16:35:35.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/1566309071 wait complete. 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.956+0000 7f4cc5c7e700 1 Processor -- start 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 -- start start 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 0x7f4cc01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc0199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cc0104f30 0x7f4cc01a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010edc0 con 0x7f4cc01045e0 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010ec40 con 0x7f4cc0104f30 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cc5c7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010ef40 con 0x7f4cc01036e0 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc0199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:35.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbf7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 0x7f4cc01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc0199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46742/0 (socket says 172.21.15.112:46742) 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 -- 172.21.15.112:0/4210616126 learned_addr learned my addr 172.21.15.112:0/4210616126 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbffff700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cc0104f30 0x7f4cc01a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 -- 172.21.15.112:0/4210616126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 msgr2=0x7f4cc01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 0x7f4cc01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 -- 172.21.15.112:0/4210616126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cc0104f30 msgr2=0x7f4cc01a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cc0104f30 0x7f4cc01a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbeffd700 1 -- 172.21.15.112:0/4210616126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc01023c0 con 0x7f4cc01045e0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.957+0000 7f4cbf7fe700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 0x7f4cc01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.958+0000 7f4cbeffd700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc0199e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4000c00 tx=0x7f4cb400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.958+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4027070 con 0x7f4cc01045e0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.958+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc0102650 con 0x7f4cc01045e0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.958+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb4022920 con 0x7f4cc01045e0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.958+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4009e60 con 0x7f4cc01045e0 2024-04-25T16:35:35.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.958+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc0102b80 con 0x7f4cc01045e0 2024-04-25T16:35:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.959+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc0051190 con 0x7f4cc01045e0 2024-04-25T16:35:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.960+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f4cb402a070 con 0x7f4cc01045e0 2024-04-25T16:35:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.961+0000 7f4cbcff9700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4ca80afb80 0x7f4ca80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:35.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.961+0000 7f4cbf7fe700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4ca80afb80 0x7f4ca80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.961+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f4cb40df460 con 0x7f4cc01045e0 2024-04-25T16:35:35.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.961+0000 7f4cbf7fe700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4ca80afb80 0x7f4ca80b2090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0104570 tx=0x7f4cb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:35.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:35.966+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cb4013070 con 0x7f4cc01045e0 2024-04-25T16:35:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.165+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f4cc002e6a0 con 0x7f4cc01045e0 2024-04-25T16:35:36.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.165+0000 7f4cbcff9700 1 -- 172.21.15.112:0/4210616126 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f4cb40a8570 con 0x7f4cc01045e0 2024-04-25T16:35:36.165 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:36.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.167+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4ca80afb80 msgr2=0x7f4ca80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.167+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4ca80afb80 0x7f4ca80b2090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0104570 tx=0x7f4cb0009260 comp rx=0 tx=0).stop 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.167+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 msgr2=0x7f4cc0199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.167+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc0199e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4000c00 tx=0x7f4cb400fa00 comp rx=0 tx=0).stop 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 shutdown_connections 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4ca80afb80 0x7f4ca80b2090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cc0104f30 0x7f4cc01a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cc01036e0 0x7f4cc01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 --2- 172.21.15.112:0/4210616126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cc01045e0 0x7f4cc0199e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 >> 172.21.15.112:0/4210616126 conn(0x7f4cc00fded0 msgr2=0x7f4cc0107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 shutdown_connections 2024-04-25T16:35:36.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:36.168+0000 7f4cc5c7e700 1 -- 172.21.15.112:0/4210616126 wait complete. 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: purged_snaps scrub starts 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: purged_snaps scrub ok 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] boot 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T16:35:36.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:36 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: purged_snaps scrub starts 2024-04-25T16:35:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: purged_snaps scrub ok 2024-04-25T16:35:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] boot 2024-04-25T16:35:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T16:35:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:36 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:36.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: purged_snaps scrub starts 2024-04-25T16:35:36.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: purged_snaps scrub ok 2024-04-25T16:35:36.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-04-25T16:35:36.531 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:36.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: osd.11 [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] boot 2024-04-25T16:35:36.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: osdmap e70: 12 total, 12 up, 12 in 2024-04-25T16:35:36.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:36 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:35:36.954 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1714062935,"num_in_osds":12,"osd_in_since":1714062915,"num_remapped_pgs":0} 2024-04-25T16:35:36.954 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T16:35:37.196 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:35:37.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4210616126' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:37.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:37.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:37 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4210616126' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.318 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:37.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:37.319 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:37 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: osdmap e71: 12 total, 12 up, 12 in 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4210616126' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:37.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:37 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:38.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:38 smithi112 ceph-mon[29122]: Detected new or changed devices on smithi133 2024-04-25T16:35:38.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:38 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi133 to 4272M 2024-04-25T16:35:38.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:38 smithi112 ceph-mon[29122]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:38 smithi133 ceph-mon[32691]: Detected new or changed devices on smithi133 2024-04-25T16:35:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:38 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi133 to 4272M 2024-04-25T16:35:38.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:38 smithi133 ceph-mon[32691]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:38.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:38 smithi116 ceph-mon[32712]: Detected new or changed devices on smithi133 2024-04-25T16:35:38.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:38 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi133 to 4272M 2024-04-25T16:35:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:38 smithi116 ceph-mon[32712]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:39.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 -- 172.21.15.112:0/465398145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 msgr2=0x7f1d58103500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 --2- 172.21.15.112:0/465398145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d58103500 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009990 tx=0x7f1d48009ca0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 -- 172.21.15.112:0/465398145 shutdown_connections 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 --2- 172.21.15.112:0/465398145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d58108540 0x7f1d58108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 --2- 172.21.15.112:0/465398145 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d58103500 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 --2- 172.21.15.112:0/465398145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d58103a40 0x7f1d58107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 -- 172.21.15.112:0/465398145 >> 172.21.15.112:0/465398145 conn(0x7f1d580fecb0 msgr2=0x7f1d581010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:39.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 -- 172.21.15.112:0/465398145 shutdown_connections 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.045+0000 7f1d5d409700 1 -- 172.21.15.112:0/465398145 wait complete. 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 Processor -- start 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 -- start start 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d581a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d58103a40 0x7f1d581a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d58108540 0x7f1d581a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5811a8e0 con 0x7f1d581030f0 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5811a760 con 0x7f1d58108540 2024-04-25T16:35:39.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d5d409700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5811aa60 con 0x7f1d58103a40 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d581a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d581a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46768/0 (socket says 172.21.15.112:46768) 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 -- 172.21.15.112:0/1553105881 learned_addr learned my addr 172.21.15.112:0/1553105881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d577fe700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d58108540 0x7f1d581a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 -- 172.21.15.112:0/1553105881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d58103a40 msgr2=0x7f1d581a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d58103a40 0x7f1d581a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 -- 172.21.15.112:0/1553105881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d58108540 msgr2=0x7f1d581a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d58108540 0x7f1d581a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.046+0000 7f1d56ffd700 1 -- 172.21.15.112:0/1553105881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d581a6120 con 0x7f1d581030f0 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.047+0000 7f1d56ffd700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d581a1250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d48000c00 tx=0x7f1d480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:39.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.047+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48027070 con 0x7f1d581030f0 2024-04-25T16:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.047+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d48003fa0 con 0x7f1d581030f0 2024-04-25T16:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.047+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d581a63b0 con 0x7f1d581030f0 2024-04-25T16:35:39.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.047+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48022db0 con 0x7f1d581030f0 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.047+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d581a6890 con 0x7f1d581030f0 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.048+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d58051190 con 0x7f1d581030f0 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.049+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f1d4802a070 con 0x7f1d581030f0 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.050+0000 7f1d3ffff700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1d400afb80 0x7f1d400b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.051+0000 7f1d567fc700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1d400afb80 0x7f1d400b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.051+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1d480df970 con 0x7f1d581030f0 2024-04-25T16:35:39.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.051+0000 7f1d567fc700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1d400afb80 0x7f1d400b2090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c005950 tx=0x7f1d4c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:39.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.054+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d48013070 con 0x7f1d581030f0 2024-04-25T16:35:39.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.258+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d581add30 con 0x7f1d581030f0 2024-04-25T16:35:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.259+0000 7f1d3ffff700 1 -- 172.21.15.112:0/1553105881 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18536 (secure 0 0 0) 0x7f1d480a89e0 con 0x7f1d581030f0 2024-04-25T16:35:39.258 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:39.259 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:35:36.145733+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1d400afb80 msgr2=0x7f1d400b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1d400afb80 0x7f1d400b2090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c005950 tx=0x7f1d4c0058e0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 msgr2=0x7f1d581a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d581a1250 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1d48000c00 tx=0x7f1d480047b0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 shutdown_connections 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1d400afb80 0x7f1d400b2090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d58108540 0x7f1d581a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d581030f0 0x7f1d581a1250 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 --2- 172.21.15.112:0/1553105881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d58103a40 0x7f1d581a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 >> 172.21.15.112:0/1553105881 conn(0x7f1d580fecb0 msgr2=0x7f1d58105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 shutdown_connections 2024-04-25T16:35:39.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:39.262+0000 7f1d5d409700 1 -- 172.21.15.112:0/1553105881 wait complete. 2024-04-25T16:35:40.070 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:35:40.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T16:35:40.316 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:35:40.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:40 smithi112 ceph-mon[29122]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:40.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:40 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1553105881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:35:40.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:40 smithi133 ceph-mon[32691]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:40.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:40 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1553105881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:35:40.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:40 smithi116 ceph-mon[32712]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:40.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:40 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1553105881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:35:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3545630743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 msgr2=0x7fdb281057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:42.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3545630743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281057a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009990 tx=0x7fdb24009ca0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3545630743 shutdown_connections 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3545630743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb2810e7c0 0x7fdb28110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3545630743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281057a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3545630743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 0x7fdb2810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3545630743 >> 172.21.15.112:0/3545630743 conn(0x7fdb280fec90 msgr2=0x7fdb281010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3545630743 shutdown_connections 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.114+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3545630743 wait complete. 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 Processor -- start 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 -- start start 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 0x7fdb281a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:42.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb2810e7c0 0x7fdb2819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28114310 con 0x7fdb28105390 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28114190 con 0x7fdb2810e7c0 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2f3f7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28114490 con 0x7fdb28105ce0 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2d193700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2d193700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46800/0 (socket says 172.21.15.112:46800) 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.115+0000 7fdb2d193700 1 -- 172.21.15.112:0/3468293951 learned_addr learned my addr 172.21.15.112:0/3468293951 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2c992700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 0x7fdb281a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d994700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb2810e7c0 0x7fdb2819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d193700 1 -- 172.21.15.112:0/3468293951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 msgr2=0x7fdb281a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d193700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 0x7fdb281a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d193700 1 -- 172.21.15.112:0/3468293951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb2810e7c0 msgr2=0x7fdb2819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d193700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb2810e7c0 0x7fdb2819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d193700 1 -- 172.21.15.112:0/3468293951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2819b1b0 con 0x7fdb28105390 2024-04-25T16:35:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2c992700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 0x7fdb281a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:35:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2d193700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281a0890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000c00 tx=0x7fdb240047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24027070 con 0x7fdb28105390 2024-04-25T16:35:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb24003fa0 con 0x7fdb28105390 2024-04-25T16:35:42.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2819b440 con 0x7fdb28105390 2024-04-25T16:35:42.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24022db0 con 0x7fdb28105390 2024-04-25T16:35:42.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.116+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb281a7740 con 0x7fdb28105390 2024-04-25T16:35:42.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.117+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb28051190 con 0x7fdb28105390 2024-04-25T16:35:42.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.118+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fdb2402a070 con 0x7fdb28105390 2024-04-25T16:35:42.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.119+0000 7fdb1e7fc700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdb140afb80 0x7fdb140b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:42.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.119+0000 7fdb2c992700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdb140afb80 0x7fdb140b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:42.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.119+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fdb240df9d0 con 0x7fdb28105390 2024-04-25T16:35:42.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.120+0000 7fdb2c992700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdb140afb80 0x7fdb140b2090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005950 tx=0x7fdb180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:42.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.123+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb24013070 con 0x7fdb28105390 2024-04-25T16:35:42.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.320+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb2802e510 con 0x7fdb28105390 2024-04-25T16:35:42.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.321+0000 7fdb1e7fc700 1 -- 172.21.15.112:0/3468293951 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7fdb240a8a50 con 0x7fdb28105390 2024-04-25T16:35:42.320 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdb140afb80 msgr2=0x7fdb140b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdb140afb80 0x7fdb140b2090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb18005950 tx=0x7fdb180058e0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 msgr2=0x7fdb281a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281a0890 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb24000c00 tx=0x7fdb240047b0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 shutdown_connections 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdb140afb80 0x7fdb140b2090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb2810e7c0 0x7fdb2819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb28105390 0x7fdb281a0890 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 --2- 172.21.15.112:0/3468293951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb28105ce0 0x7fdb281a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:42.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 >> 172.21.15.112:0/3468293951 conn(0x7fdb280fec90 msgr2=0x7fdb280ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:42.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 shutdown_connections 2024-04-25T16:35:42.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:42.323+0000 7fdb2f3f7700 1 -- 172.21.15.112:0/3468293951 wait complete. 2024-04-25T16:35:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:42 smithi133 ceph-mon[32691]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:42 smithi116 ceph-mon[32712]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:42 smithi112 ceph-mon[29122]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.a on smithi112 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.d on smithi112 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.g on smithi112 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.j on smithi112 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.b on smithi116 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.e on smithi116 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.h on smithi116 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.k on smithi116 2024-04-25T16:35:43.103 INFO:tasks.cephadm:Adding mds.c on smithi133 2024-04-25T16:35:43.104 INFO:tasks.cephadm:Adding mds.f on smithi133 2024-04-25T16:35:43.104 INFO:tasks.cephadm:Adding mds.i on smithi133 2024-04-25T16:35:43.104 INFO:tasks.cephadm:Adding mds.l on smithi133 2024-04-25T16:35:43.104 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l' 2024-04-25T16:35:43.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:43 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3468293951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:35:43.355 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:35:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:43 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3468293951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:35:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:43 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3468293951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:35:44.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:44 smithi133 ceph-mon[32691]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:44.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:44 smithi116 ceph-mon[32712]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:44.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:44 smithi112 ceph-mon[29122]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:45.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 -- 172.21.15.133:0/176369034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 msgr2=0x7f1b54106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:45.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 --2- 172.21.15.133:0/176369034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54106dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400b0a0 tx=0x7f1b4400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 -- 172.21.15.133:0/176369034 shutdown_connections 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 --2- 172.21.15.133:0/176369034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b54109ec0 0x7f1b5410c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 --2- 172.21.15.133:0/176369034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54106dc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 --2- 172.21.15.133:0/176369034 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54107390 0x7f1b54109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 -- 172.21.15.133:0/176369034 >> 172.21.15.133:0/176369034 conn(0x7f1b540fe750 msgr2=0x7f1b54100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 -- 172.21.15.133:0/176369034 shutdown_connections 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.035+0000 7f1b59808700 1 -- 172.21.15.133:0/176369034 wait complete. 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 Processor -- start 2024-04-25T16:35:45.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 -- start start 2024-04-25T16:35:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b54107390 0x7f1b54102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54109ec0 0x7f1b54103200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410fcc0 con 0x7f1b54109ec0 2024-04-25T16:35:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410fb40 con 0x7f1b54107390 2024-04-25T16:35:45.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b59808700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5410fe40 con 0x7f1b5406c160 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40646/0 (socket says 172.21.15.133:40646) 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.036+0000 7f1b52ffd700 1 -- 172.21.15.133:0/2539017724 learned_addr learned my addr 172.21.15.133:0/2539017724 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b4a7fc700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b54107390 0x7f1b54102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b537fe700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54109ec0 0x7f1b54103200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b52ffd700 1 -- 172.21.15.133:0/2539017724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b54107390 msgr2=0x7f1b54102cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b52ffd700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b54107390 0x7f1b54102cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b52ffd700 1 -- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54109ec0 msgr2=0x7f1b54103200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b52ffd700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54109ec0 0x7f1b54103200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b52ffd700 1 -- 172.21.15.133:0/2539017724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b54103960 con 0x7f1b5406c160 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b537fe700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54109ec0 0x7f1b54103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b52ffd700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54104610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b440095c0 tx=0x7f1b44004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:45.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44021070 con 0x7f1b5406c160 2024-04-25T16:35:45.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b4401d900 con 0x7f1b5406c160 2024-04-25T16:35:45.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b541a48e0 con 0x7f1b5406c160 2024-04-25T16:35:45.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440052a0 con 0x7f1b5406c160 2024-04-25T16:35:45.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.037+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b541a4dc0 con 0x7f1b5406c160 2024-04-25T16:35:45.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.038+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b54051190 con 0x7f1b5406c160 2024-04-25T16:35:45.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.040+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f1b44024070 con 0x7f1b5406c160 2024-04-25T16:35:45.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.040+0000 7f1b58806700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1b400afb80 0x7f1b400b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:45.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.040+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f1b4400f020 con 0x7f1b5406c160 2024-04-25T16:35:45.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.041+0000 7f1b4a7fc700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1b400afb80 0x7f1b400b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:45.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.041+0000 7f1b4a7fc700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1b400afb80 0x7f1b400b2090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c005950 tx=0x7f1b3c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:45.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.044+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b4402b020 con 0x7f1b5406c160 2024-04-25T16:35:45.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.267+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}) -- 0x7f1b54063cd0 con 0x7f1b400afb80 2024-04-25T16:35:45.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.275+0000 7f1b58806700 1 -- 172.21.15.133:0/2539017724 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f1b54063cd0 con 0x7f1b400afb80 2024-04-25T16:35:45.274 INFO:teuthology.orchestra.run.smithi133.stdout:Scheduled mds.all update... 2024-04-25T16:35:45.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.277+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1b400afb80 msgr2=0x7f1b400b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:45.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1b400afb80 0x7f1b400b2090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c005950 tx=0x7f1b3c0058e0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 msgr2=0x7f1b54104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54104610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b440095c0 tx=0x7f1b44004ef0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 shutdown_connections 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b54109ec0 0x7f1b54103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b5406c160 0x7f1b54104610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f1b400afb80 0x7f1b400b2090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 --2- 172.21.15.133:0/2539017724 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b54107390 0x7f1b54102cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 >> 172.21.15.133:0/2539017724 conn(0x7f1b540fe750 msgr2=0x7f1b541057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 shutdown_connections 2024-04-25T16:35:45.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-04-25T16:35:45.278+0000 7f1b59808700 1 -- 172.21.15.133:0/2539017724 wait complete. 2024-04-25T16:35:46.010 DEBUG:teuthology.orchestra.run.smithi112:mds.a> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T16:35:46.013 DEBUG:teuthology.orchestra.run.smithi112:mds.d> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T16:35:46.056 DEBUG:teuthology.orchestra.run.smithi112:mds.g> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T16:35:46.100 DEBUG:teuthology.orchestra.run.smithi112:mds.j> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T16:35:46.143 DEBUG:teuthology.orchestra.run.smithi116:mds.b> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T16:35:46.146 DEBUG:teuthology.orchestra.run.smithi116:mds.e> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T16:35:46.189 DEBUG:teuthology.orchestra.run.smithi116:mds.h> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T16:35:46.231 DEBUG:teuthology.orchestra.run.smithi116:mds.k> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T16:35:46.275 DEBUG:teuthology.orchestra.run.smithi133:mds.c> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T16:35:46.277 DEBUG:teuthology.orchestra.run.smithi133:mds.f> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T16:35:46.299 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:46 smithi133 ceph-mon[32691]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:46.299 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:46.299 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:46 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:46.300 DEBUG:teuthology.orchestra.run.smithi133:mds.i> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T16:35:46.342 DEBUG:teuthology.orchestra.run.smithi133:mds.l> sudo journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T16:35:46.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:46 smithi112 ceph-mon[29122]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:46.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:46.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:46 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:46.385 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T16:35:46.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:35:46.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 -- 172.21.15.112:0/983870415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841029e0 msgr2=0x7f32841036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 --2- 172.21.15.112:0/983870415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841029e0 0x7f32841036e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3274009900 tx=0x7f327402a410 comp rx=0 tx=0).stop 2024-04-25T16:35:46.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 -- 172.21.15.112:0/983870415 shutdown_connections 2024-04-25T16:35:46.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 --2- 172.21.15.112:0/983870415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3284103c20 0x7f3284104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 --2- 172.21.15.112:0/983870415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3284108920 0x7f328410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 --2- 172.21.15.112:0/983870415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f32841029e0 0x7f32841036e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 -- 172.21.15.112:0/983870415 >> 172.21.15.112:0/983870415 conn(0x7f32840fdbd0 msgr2=0x7f3284100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 -- 172.21.15.112:0/983870415 shutdown_connections 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.504+0000 7f328910d640 1 -- 172.21.15.112:0/983870415 wait complete. 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 Processor -- start 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 -- start start 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32841029e0 0x7f32841991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3284103c20 0x7f32841996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 0x7f32841a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f328410e4e0 con 0x7f3284108920 2024-04-25T16:35:46.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f328410e360 con 0x7f3284103c20 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328910d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f328410e660 con 0x7f32841029e0 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 0x7f32841a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 0x7f32841a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54592/0 (socket says 172.21.15.112:54592) 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328359e640 1 -- 172.21.15.112:0/3529265008 learned_addr learned my addr 172.21.15.112:0/3529265008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f3282d9d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32841029e0 0x7f32841991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.505+0000 7f328259c640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3284103c20 0x7f32841996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328359e640 1 -- 172.21.15.112:0/3529265008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32841029e0 msgr2=0x7f32841991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328359e640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32841029e0 0x7f32841991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328359e640 1 -- 172.21.15.112:0/3529265008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3284103c20 msgr2=0x7f32841996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328359e640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3284103c20 0x7f32841996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328359e640 1 -- 172.21.15.112:0/3529265008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32841a0ce0 con 0x7f3284108920 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328359e640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 0x7f32841a0670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f327c00ebb0 tx=0x7f327c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:46.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c007ca0 con 0x7f3284108920 2024-04-25T16:35:46.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f327c007e40 con 0x7f3284108920 2024-04-25T16:35:46.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3284074dd0 con 0x7f3284108920 2024-04-25T16:35:46.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c0057d0 con 0x7f3284108920 2024-04-25T16:35:46.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.506+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32840752d0 con 0x7f3284108920 2024-04-25T16:35:46.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.507+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f328406a190 con 0x7f3284108920 2024-04-25T16:35:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:46 smithi116 ceph-mon[32712]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:46.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:46 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:35:46.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.508+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f327c004a40 con 0x7f3284108920 2024-04-25T16:35:46.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.509+0000 7f326bfff640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f326c0afaf0 0x7f326c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.509+0000 7f3282d9d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f326c0afaf0 0x7f326c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.509+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f327c0e0120 con 0x7f3284108920 2024-04-25T16:35:46.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.509+0000 7f3282d9d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f326c0afaf0 0x7f326c0b2000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3274005ab0 tx=0x7f3274031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:46.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.512+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f327c0a8b50 con 0x7f3284108920 2024-04-25T16:35:46.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.690+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32841073f0 con 0x7f3284108920 2024-04-25T16:35:46.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.691+0000 7f326bfff640 1 -- 172.21.15.112:0/3529265008 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18536 (secure 0 0 0) 0x7f327c00e0d0 con 0x7f3284108920 2024-04-25T16:35:46.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:46.690 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:35:36.145733+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f326c0afaf0 msgr2=0x7f326c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f326c0afaf0 0x7f326c0b2000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3274005ab0 tx=0x7f3274031400 comp rx=0 tx=0).stop 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 msgr2=0x7f32841a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 0x7f32841a0670 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f327c00ebb0 tx=0x7f327c007590 comp rx=0 tx=0).stop 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 shutdown_connections 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3284108920 0x7f32841a0670 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3284103c20 0x7f32841996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f326c0afaf0 0x7f326c0b2000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 --2- 172.21.15.112:0/3529265008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32841029e0 0x7f32841991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 >> 172.21.15.112:0/3529265008 conn(0x7f32840fdbd0 msgr2=0x7f3284104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 shutdown_connections 2024-04-25T16:35:46.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.693+0000 7f328910d640 1 -- 172.21.15.112:0/3529265008 wait complete. 2024-04-25T16:35:46.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:35:46.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- 172.21.15.112:0/3132504494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c103c40 msgr2=0x7f342c104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- 172.21.15.112:0/3132504494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c103c40 0x7f342c104050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3418009900 tx=0x7f341802a410 comp rx=0 tx=0).stop 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- 172.21.15.112:0/3132504494 shutdown_connections 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- 172.21.15.112:0/3132504494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c108940 0x7f342c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- 172.21.15.112:0/3132504494 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c103c40 0x7f342c104050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- 172.21.15.112:0/3132504494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f342c102a00 0x7f342c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- 172.21.15.112:0/3132504494 >> 172.21.15.112:0/3132504494 conn(0x7f342c0fdbf0 msgr2=0x7f342c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- 172.21.15.112:0/3132504494 shutdown_connections 2024-04-25T16:35:46.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- 172.21.15.112:0/3132504494 wait complete. 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 Processor -- start 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- start start 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c102a00 0x7f342c075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f342c103c40 0x7f342c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 0x7f342c1a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f342c10e500 con 0x7f342c108940 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f342c10e380 con 0x7f342c103c40 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.815+0000 7f3432a6e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f342c10e680 con 0x7f342c102a00 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 0x7f342c1a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 0x7f342c1a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54606/0 (socket says 172.21.15.112:54606) 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 -- 172.21.15.112:0/380152442 learned_addr learned my addr 172.21.15.112:0/380152442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343080a640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c102a00 0x7f342c075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f342bfff640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f342c103c40 0x7f342c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 -- 172.21.15.112:0/380152442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c102a00 msgr2=0x7f342c075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c102a00 0x7f342c075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 -- 172.21.15.112:0/380152442 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f342c103c40 msgr2=0x7f342c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f342c103c40 0x7f342c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 -- 172.21.15.112:0/380152442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1a1d00 con 0x7f342c108940 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f343100b640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 0x7f342c1a14e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3424009fd0 tx=0x7f342400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424017070 con 0x7f342c108940 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3424003e20 con 0x7f342c108940 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1a1ff0 con 0x7f342c108940 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424010460 con 0x7f342c108940 2024-04-25T16:35:46.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.816+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a2450 con 0x7f342c108940 2024-04-25T16:35:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.818+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c107750 con 0x7f342c108940 2024-04-25T16:35:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.819+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f3424003970 con 0x7f342c108940 2024-04-25T16:35:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.819+0000 7f3429ffb640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f341c0afaf0 0x7f341c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.819+0000 7f343080a640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f341c0afaf0 0x7f341c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:46.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.820+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f34240de400 con 0x7f342c108940 2024-04-25T16:35:46.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.820+0000 7f343080a640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f341c0afaf0 0x7f341c0b2000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3420005fd0 tx=0x7f342000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:46.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:46.824+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f34240a6e30 con 0x7f342c108940 2024-04-25T16:35:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.002+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f342c02dda0 con 0x7f342c108940 2024-04-25T16:35:47.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.002+0000 7f3429ffb640 1 -- 172.21.15.112:0/380152442 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f34240ab130 con 0x7f342c108940 2024-04-25T16:35:47.001 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:35:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f341c0afaf0 msgr2=0x7f341c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f341c0afaf0 0x7f341c0b2000 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3420005fd0 tx=0x7f342000b410 comp rx=0 tx=0).stop 2024-04-25T16:35:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 msgr2=0x7f342c1a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 0x7f342c1a14e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3424009fd0 tx=0x7f342400b260 comp rx=0 tx=0).stop 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 shutdown_connections 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f342c108940 0x7f342c1a14e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f342c103c40 0x7f342c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f341c0afaf0 0x7f341c0b2000 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 --2- 172.21.15.112:0/380152442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f342c102a00 0x7f342c075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 >> 172.21.15.112:0/380152442 conn(0x7f342c0fdbf0 msgr2=0x7f342c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 shutdown_connections 2024-04-25T16:35:47.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.004+0000 7f3432a6e640 1 -- 172.21.15.112:0/380152442 wait complete. 2024-04-25T16:35:47.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 -- 172.21.15.112:0/2919094710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde041098b0 msgr2=0x7fde0410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 --2- 172.21.15.112:0/2919094710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde041098b0 0x7fde0410bcd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009ca0 tx=0x7fddfc02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 -- 172.21.15.112:0/2919094710 shutdown_connections 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 --2- 172.21.15.112:0/2919094710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde04106f50 0x7fde04109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 --2- 172.21.15.112:0/2919094710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde041098b0 0x7fde0410bcd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 --2- 172.21.15.112:0/2919094710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde041045f0 0x7fde04106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 -- 172.21.15.112:0/2919094710 >> 172.21.15.112:0/2919094710 conn(0x7fde040fdbf0 msgr2=0x7fde041000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:47.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 -- 172.21.15.112:0/2919094710 shutdown_connections 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.123+0000 7fde0a220640 1 -- 172.21.15.112:0/2919094710 wait complete. 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 Processor -- start 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 -- start start 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde041045f0 0x7fde0419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04106f50 0x7fde0419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 0x7fde041a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde0410f610 con 0x7fde04106f50 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fde0410f490 con 0x7fde041045f0 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde0a220640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fde0410f790 con 0x7fde041098b0 2024-04-25T16:35:47.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde03fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 0x7fde041a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde041045f0 0x7fde0419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde03fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 0x7fde041a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:43258/0 (socket says 172.21.15.112:43258) 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.124+0000 7fde03fff640 1 -- 172.21.15.112:0/3913841576 learned_addr learned my addr 172.21.15.112:0/3913841576 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde02ffd640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04106f50 0x7fde0419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde03fff640 1 -- 172.21.15.112:0/3913841576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde041045f0 msgr2=0x7fde0419a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde03fff640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde041045f0 0x7fde0419a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde03fff640 1 -- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04106f50 msgr2=0x7fde0419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde03fff640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04106f50 0x7fde0419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde03fff640 1 -- 172.21.15.112:0/3913841576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde041a1c90 con 0x7fde041098b0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde03fff640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 0x7fde041a1620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009430 tx=0x7fddfc003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc02f380 con 0x7fde041098b0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddfc0317c0 con 0x7fde041098b0 2024-04-25T16:35:47.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde041a1f20 con 0x7fde041098b0 2024-04-25T16:35:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc02fcd0 con 0x7fde041098b0 2024-04-25T16:35:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.125+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde041a2380 con 0x7fde041098b0 2024-04-25T16:35:47.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.126+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde0406a190 con 0x7fde041098b0 2024-04-25T16:35:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.128+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fddfc004040 con 0x7fde041098b0 2024-04-25T16:35:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.128+0000 7fde00ff9640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fddec0afc90 0x7fddec0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.128+0000 7fde037fe640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fddec0afc90 0x7fddec0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.128+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fddfc0fff60 con 0x7fde041098b0 2024-04-25T16:35:47.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.129+0000 7fde037fe640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fddec0afc90 0x7fddec0b21a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde040757f0 tx=0x7fddf800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:47.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.132+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fddfc0c8fc0 con 0x7fde041098b0 2024-04-25T16:35:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:35:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-04-25T16:35:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:47.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: Deploying daemon mds.l on smithi133 2024-04-25T16:35:47.347 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3529265008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:35:47.348 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:47 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/380152442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:35:47.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.347+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde0402dc10 con 0x7fde041098b0 2024-04-25T16:35:47.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.348+0000 7fde00ff9640 1 -- 172.21.15.112:0/3913841576 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fddfc0098b0 con 0x7fde041098b0 2024-04-25T16:35:47.350 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:47.350 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T16:35:47.352 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 1 2024-04-25T16:35:47.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fddec0afc90 msgr2=0x7fddec0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fddec0afc90 0x7fddec0b21a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde040757f0 tx=0x7fddf800b390 comp rx=0 tx=0).stop 2024-04-25T16:35:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 msgr2=0x7fde041a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 0x7fde041a1620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009430 tx=0x7fddfc003ba0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 shutdown_connections 2024-04-25T16:35:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fddec0afc90 0x7fddec0b21a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde041098b0 0x7fde041a1620 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde04106f50 0x7fde0419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 --2- 172.21.15.112:0/3913841576 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde041045f0 0x7fde0419a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 >> 172.21.15.112:0/3913841576 conn(0x7fde040fdbf0 msgr2=0x7fde041050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 shutdown_connections 2024-04-25T16:35:47.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:47.350+0000 7fde0a220640 1 -- 172.21.15.112:0/3913841576 wait complete. 2024-04-25T16:35:47.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:35:47.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-04-25T16:35:47.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:47.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:47.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:47.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:47.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:47.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:47.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: Deploying daemon mds.l on smithi133 2024-04-25T16:35:47.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3529265008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:35:47.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:47 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/380152442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:35:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: Saving service mds.all spec with placement smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: Deploying daemon mds.l on smithi133 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3529265008' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:35:47.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:47 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/380152442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:35:48.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:48 smithi133 ceph-mon[32691]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:48.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:48 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3913841576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:48.450 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 systemd[1]: Starting Ceph mds.l for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:48 smithi116 ceph-mon[32712]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:48.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:48 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3913841576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:48.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:48 smithi112 ceph-mon[29122]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:48.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:48 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3913841576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:48.851 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 podman[62819]: 2024-04-25 16:35:48.464156558 +0000 UTC m=+0.017992200 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:35:48.851 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 podman[62819]: 2024-04-25 16:35:48.584284515 +0000 UTC m=+0.138120154 container create 13692d82e059f43c61cc07f71241e1a2a462952e905833ca710c57f5656db700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:35:48.851 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 podman[62819]: 2024-04-25 16:35:48.853667191 +0000 UTC m=+0.407502832 container init 13692d82e059f43c61cc07f71241e1a2a462952e905833ca710c57f5656db700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T16:35:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 podman[62819]: 2024-04-25 16:35:48.856351745 +0000 UTC m=+0.410187386 container start 13692d82e059f43c61cc07f71241e1a2a462952e905833ca710c57f5656db700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:35:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 ceph-mds[62839]: -- 172.21.15.133:0/3737283473 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccff5aba40 con 0x55ccff5f3180 2024-04-25T16:35:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l[62835]: starting mds.l at 2024-04-25T16:35:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 bash[62819]: 13692d82e059f43c61cc07f71241e1a2a462952e905833ca710c57f5656db700 2024-04-25T16:35:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:35:48 smithi133 systemd[1]: Started Ceph mds.l for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:35:49.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:35:49.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 -- 172.21.15.112:0/798730708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8228108920 msgr2=0x7f822810ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:49.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 --2- 172.21.15.112:0/798730708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8228108920 0x7f822810ad40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8220009ca0 tx=0x7f822002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 -- 172.21.15.112:0/798730708 shutdown_connections 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 --2- 172.21.15.112:0/798730708 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8228108920 0x7f822810ad40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 --2- 172.21.15.112:0/798730708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8228103c20 0x7f8228104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 --2- 172.21.15.112:0/798730708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82281029e0 0x7f82281036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 -- 172.21.15.112:0/798730708 >> 172.21.15.112:0/798730708 conn(0x7f82280fdbf0 msgr2=0x7f8228100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 -- 172.21.15.112:0/798730708 shutdown_connections 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.484+0000 7f822d1b2640 1 -- 172.21.15.112:0/798730708 wait complete. 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 Processor -- start 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 -- start start 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 0x7f82281991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:49.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8228103c20 0x7f82281996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8228108920 0x7f82281a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f822810e510 con 0x7f82281029e0 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f822810e390 con 0x7f8228108920 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.485+0000 7f822d1b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f822810e690 con 0x7f8228103c20 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 0x7f82281991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 0x7f82281991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54632/0 (socket says 172.21.15.112:54632) 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 -- 172.21.15.112:0/2549879987 learned_addr learned my addr 172.21.15.112:0/2549879987 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f821ffff640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8228103c20 0x7f82281996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f822759e640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8228108920 0x7f82281a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 -- 172.21.15.112:0/2549879987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8228103c20 msgr2=0x7f82281996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8228103c20 0x7f82281996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 -- 172.21.15.112:0/2549879987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8228108920 msgr2=0x7f82281a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:49.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8228108920 0x7f82281a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 -- 172.21.15.112:0/2549879987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82281a0c70 con 0x7f82281029e0 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8226d9d640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 0x7f82281991a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f821800e9f0 tx=0x7f821800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800ccc0 con 0x7f82281029e0 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82281a0f60 con 0x7f82281029e0 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82180035f0 con 0x7f82281029e0 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800ce60 con 0x7f82281029e0 2024-04-25T16:35:49.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.486+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a13c0 con 0x7f82281029e0 2024-04-25T16:35:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.487+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8228107730 con 0x7f82281029e0 2024-04-25T16:35:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.489+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f8218003aa0 con 0x7f82281029e0 2024-04-25T16:35:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.489+0000 7f8224d99640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82140afaf0 0x7f82140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.489+0000 7f821ffff640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82140afaf0 0x7f82140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.489+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f82180debb0 con 0x7f82281029e0 2024-04-25T16:35:49.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.490+0000 7f821ffff640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82140afaf0 0x7f82140b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f822819a240 tx=0x7f8210005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:49.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.493+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f82180a7c30 con 0x7f82281029e0 2024-04-25T16:35:49.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.718+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f822802dda0 con 0x7f82281029e0 2024-04-25T16:35:49.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.718+0000 7f8224d99640 1 -- 172.21.15.112:0/2549879987 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f82180a7530 con 0x7f82281029e0 2024-04-25T16:35:49.717 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:49.717 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T16:35:49.718 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-04-25T16:35:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82140afaf0 msgr2=0x7f82140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82140afaf0 0x7f82140b2000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f822819a240 tx=0x7f8210005f90 comp rx=0 tx=0).stop 2024-04-25T16:35:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 msgr2=0x7f82281991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 0x7f82281991a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f821800e9f0 tx=0x7f821800eeb0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 shutdown_connections 2024-04-25T16:35:49.719 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82140afaf0 0x7f82140b2000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8228103c20 0x7f82281996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8228108920 0x7f82281a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 --2- 172.21.15.112:0/2549879987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82281029e0 0x7f82281991a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 >> 172.21.15.112:0/2549879987 conn(0x7f82280fdbf0 msgr2=0x7f8228104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 shutdown_connections 2024-04-25T16:35:49.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:49.720+0000 7f822d1b2640 1 -- 172.21.15.112:0/2549879987 wait complete. 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6832/2075237111,v1:172.21.15.133:6833/2075237111] up:boot 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: fsmap 1 up:standby 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: Deploying daemon mds.k on smithi116 2024-04-25T16:35:49.979 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:49.980 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:49 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2549879987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6832/2075237111,v1:172.21.15.133:6833/2075237111] up:boot 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: fsmap 1 up:standby 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:50.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:50.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:50.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:50.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: Deploying daemon mds.k on smithi116 2024-04-25T16:35:50.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:50.097 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:49 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2549879987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6832/2075237111,v1:172.21.15.133:6833/2075237111] up:boot 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: fsmap 1 up:standby 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:50.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:50.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:50.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:50.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: Deploying daemon mds.k on smithi116 2024-04-25T16:35:50.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:50.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:49 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2549879987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:51.509 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 systemd[1]: Starting Ceph mds.k for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:35:51.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:35:51.797 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 podman[62422]: 2024-04-25 16:35:51.421767958 +0000 UTC m=+0.018113624 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:35:51.797 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 podman[62422]: 2024-04-25 16:35:51.546450109 +0000 UTC m=+0.142795761 container create 627db4aa9bea5cf11ee89a487f2357e2b6b18d25d19c3082981c4ea620be34c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:35:51.797 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 podman[62422]: 2024-04-25 16:35:51.79936237 +0000 UTC m=+0.395708022 container init 627db4aa9bea5cf11ee89a487f2357e2b6b18d25d19c3082981c4ea620be34c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T16:35:51.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 -- 172.21.15.112:0/3855005372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da8103c00 msgr2=0x7f3da8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:51.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 --2- 172.21.15.112:0/3855005372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da8103c00 0x7f3da8104010 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009900 tx=0x7f3d9802a410 comp rx=0 tx=0).stop 2024-04-25T16:35:51.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 -- 172.21.15.112:0/3855005372 shutdown_connections 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 --2- 172.21.15.112:0/3855005372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8108900 0x7f3da810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 --2- 172.21.15.112:0/3855005372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da8103c00 0x7f3da8104010 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 --2- 172.21.15.112:0/3855005372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3da81029c0 0x7f3da81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 -- 172.21.15.112:0/3855005372 >> 172.21.15.112:0/3855005372 conn(0x7f3da80fdbd0 msgr2=0x7f3da8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 -- 172.21.15.112:0/3855005372 shutdown_connections 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.855+0000 7f3db03c5640 1 -- 172.21.15.112:0/3855005372 wait complete. 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 Processor -- start 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 -- start start 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 0x7f3da819a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3da8103c00 0x7f3da819a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8108900 0x7f3da81a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810e4f0 con 0x7f3da81029c0 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810e370 con 0x7f3da8103c00 2024-04-25T16:35:51.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3db03c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810e670 con 0x7f3da8108900 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 0x7f3da819a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 0x7f3da819a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54638/0 (socket says 172.21.15.112:54638) 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 -- 172.21.15.112:0/2401837581 learned_addr learned my addr 172.21.15.112:0/2401837581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dad960640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3da8103c00 0x7f3da819a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae962640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8108900 0x7f3da81a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 -- 172.21.15.112:0/2401837581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8108900 msgr2=0x7f3da81a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8108900 0x7f3da81a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 -- 172.21.15.112:0/2401837581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3da8103c00 msgr2=0x7f3da819a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3da8103c00 0x7f3da819a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.856+0000 7f3dae161640 1 -- 172.21.15.112:0/2401837581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da81a1d50 con 0x7f3da81029c0 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.857+0000 7f3dae161640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 0x7f3da819a210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e9f0 tx=0x7f3da400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.857+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400ccc0 con 0x7f3da81029c0 2024-04-25T16:35:51.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.857+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da40035f0 con 0x7f3da81029c0 2024-04-25T16:35:51.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.857+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da400ce60 con 0x7f3da81029c0 2024-04-25T16:35:51.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.857+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da81a1fe0 con 0x7f3da81029c0 2024-04-25T16:35:51.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.857+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da81a24a0 con 0x7f3da81029c0 2024-04-25T16:35:51.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.858+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da81a2760 con 0x7f3da81029c0 2024-04-25T16:35:51.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.859+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f3da4003aa0 con 0x7f3da81029c0 2024-04-25T16:35:51.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.860+0000 7f3d9f7fe640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3d940afaf0 0x7f3d940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:51.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.860+0000 7f3dad960640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3d940afaf0 0x7f3d940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:51.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.860+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3da40debc0 con 0x7f3da81029c0 2024-04-25T16:35:51.860 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.860+0000 7f3dad960640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3d940afaf0 0x7f3d940b2000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009230 tx=0x7f3d98031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:51.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:51.863+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3da40a7c40 con 0x7f3da81029c0 2024-04-25T16:35:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.093+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3da81a2930 con 0x7f3da81029c0 2024-04-25T16:35:52.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.094+0000 7f3d9f7fe640 1 -- 172.21.15.112:0/2401837581 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f3da40a7540 con 0x7f3da81029c0 2024-04-25T16:35:52.093 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:52.093 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T16:35:52.093 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-04-25T16:35:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3d940afaf0 msgr2=0x7f3d940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3d940afaf0 0x7f3d940b2000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009230 tx=0x7f3d98031400 comp rx=0 tx=0).stop 2024-04-25T16:35:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 msgr2=0x7f3da819a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:52.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 0x7f3da819a210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3da400e9f0 tx=0x7f3da400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 shutdown_connections 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3d940afaf0 0x7f3d940b2000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3da8108900 0x7f3da81a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3da8103c00 0x7f3da819a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 --2- 172.21.15.112:0/2401837581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3da81029c0 0x7f3da819a210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 >> 172.21.15.112:0/2401837581 conn(0x7f3da80fdbd0 msgr2=0x7f3da8109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 shutdown_connections 2024-04-25T16:35:52.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:52.096+0000 7f3db03c5640 1 -- 172.21.15.112:0/2401837581 wait complete. 2024-04-25T16:35:52.122 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 podman[62422]: 2024-04-25 16:35:51.80226548 +0000 UTC m=+0.398611141 container start 627db4aa9bea5cf11ee89a487f2357e2b6b18d25d19c3082981c4ea620be34c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:35:52.122 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 ceph-mds[62441]: -- 172.21.15.116:0/2385269668 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565056aaba40 con 0x565056af3180 2024-04-25T16:35:52.122 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: starting mds.k at 2024-04-25T16:35:52.122 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 bash[62422]: 627db4aa9bea5cf11ee89a487f2357e2b6b18d25d19c3082981c4ea620be34c5 2024-04-25T16:35:52.122 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:35:51 smithi116 systemd[1]: Started Ceph mds.k for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:52.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:52 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2401837581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:52.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:52.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:52.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:52 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2401837581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:52.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:52.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:52.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:52 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2401837581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:53.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:53 smithi112 ceph-mon[29122]: Deploying daemon mds.j on smithi112 2024-04-25T16:35:53.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:53 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6832/2123343031,v1:172.21.15.116:6833/2123343031] up:boot 2024-04-25T16:35:53.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:53 smithi112 ceph-mon[29122]: fsmap 2 up:standby 2024-04-25T16:35:53.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:53 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:35:53.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:53 smithi133 ceph-mon[32691]: Deploying daemon mds.j on smithi112 2024-04-25T16:35:53.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:53 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6832/2123343031,v1:172.21.15.116:6833/2123343031] up:boot 2024-04-25T16:35:53.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:53 smithi133 ceph-mon[32691]: fsmap 2 up:standby 2024-04-25T16:35:53.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:53 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:35:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:53 smithi116 ceph-mon[32712]: Deploying daemon mds.j on smithi112 2024-04-25T16:35:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:53 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6832/2123343031,v1:172.21.15.116:6833/2123343031] up:boot 2024-04-25T16:35:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:53 smithi116 ceph-mon[32712]: fsmap 2 up:standby 2024-04-25T16:35:53.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:53 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:35:54.108 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:35:54.222 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:54 smithi116 ceph-mon[32712]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 -- 172.21.15.112:0/404864355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4103c50 msgr2=0x7effd4104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 --2- 172.21.15.112:0/404864355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4103c50 0x7effd4104060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a410 comp rx=0 tx=0).stop 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 -- 172.21.15.112:0/404864355 shutdown_connections 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 --2- 172.21.15.112:0/404864355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd4108950 0x7effd410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 --2- 172.21.15.112:0/404864355 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4103c50 0x7effd4104060 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 --2- 172.21.15.112:0/404864355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effd4102a10 0x7effd4103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 -- 172.21.15.112:0/404864355 >> 172.21.15.112:0/404864355 conn(0x7effd40fdc00 msgr2=0x7effd41000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 -- 172.21.15.112:0/404864355 shutdown_connections 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 -- 172.21.15.112:0/404864355 wait complete. 2024-04-25T16:35:54.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 Processor -- start 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.264+0000 7effdb90d640 1 -- start start 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effdb90d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd4102a10 0x7effd406bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effdb90d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effd4103c50 0x7effd419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effdb90d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 0x7effd41a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effdb90d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7effd410e560 con 0x7effd4108950 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effdb90d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7effd410e3e0 con 0x7effd4103c50 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effdb90d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7effd410e6e0 con 0x7effd4102a10 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 0x7effd41a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 0x7effd41a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54660/0 (socket says 172.21.15.112:54660) 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 -- 172.21.15.112:0/1919525411 learned_addr learned my addr 172.21.15.112:0/1919525411 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd8ea8640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effd4103c50 0x7effd419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd96a9640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd4102a10 0x7effd406bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:54.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 -- 172.21.15.112:0/1919525411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd4102a10 msgr2=0x7effd406bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd4102a10 0x7effd406bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 -- 172.21.15.112:0/1919525411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effd4103c50 msgr2=0x7effd419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effd4103c50 0x7effd419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 -- 172.21.15.112:0/1919525411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd41a1db0 con 0x7effd4108950 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effd9eaa640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 0x7effd41a1590 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7effcc00eaa0 tx=0x7effcc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.265+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc00cdb0 con 0x7effd4108950 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.266+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd41a20a0 con 0x7effd4108950 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.266+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effcc005020 con 0x7effd4108950 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.266+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0116d0 con 0x7effd4108950 2024-04-25T16:35:54.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.266+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd41a25c0 con 0x7effd4108950 2024-04-25T16:35:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.267+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd4069310 con 0x7effd4108950 2024-04-25T16:35:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.268+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7effcc0051d0 con 0x7effd4108950 2024-04-25T16:35:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.269+0000 7effca7fc640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7effc00afaf0 0x7effc00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.269+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7effcc0dec90 con 0x7effd4108950 2024-04-25T16:35:54.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.269+0000 7effd96a9640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7effc00afaf0 0x7effc00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:54.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.270+0000 7effd96a9640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7effc00afaf0 0x7effc00b2000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7effd4103610 tx=0x7effd0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:54.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.274+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7effcc016030 con 0x7effd4108950 2024-04-25T16:35:54.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:54 smithi112 ceph-mon[29122]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:54.346 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 systemd[1]: Starting Ceph mds.j for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:35:54.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:54 smithi133 ceph-mon[32691]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:54.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.517+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd4107760 con 0x7effd4108950 2024-04-25T16:35:54.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.517+0000 7effca7fc640 1 -- 172.21.15.112:0/1919525411 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2224 (secure 0 0 0) 0x7effcc0a76c0 con 0x7effd4108950 2024-04-25T16:35:54.517 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:54.517 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T16:35:54.517 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-04-25T16:35:54.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7effc00afaf0 msgr2=0x7effc00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7effc00afaf0 0x7effc00b2000 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7effd4103610 tx=0x7effd0005f90 comp rx=0 tx=0).stop 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 msgr2=0x7effd41a1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 0x7effd41a1590 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7effcc00eaa0 tx=0x7effcc00ef60 comp rx=0 tx=0).stop 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 shutdown_connections 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7effd4108950 0x7effd41a1590 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7effd4103c50 0x7effd419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7effc00afaf0 0x7effc00b2000 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.519+0000 7effdb90d640 1 --2- 172.21.15.112:0/1919525411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd4102a10 0x7effd406bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.520+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 >> 172.21.15.112:0/1919525411 conn(0x7effd40fdc00 msgr2=0x7effd4104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.520+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 shutdown_connections 2024-04-25T16:35:54.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:54.520+0000 7effdb90d640 1 -- 172.21.15.112:0/1919525411 wait complete. 2024-04-25T16:35:54.686 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 podman[70552]: 2024-04-25 16:35:54.336353629 +0000 UTC m=+0.024967192 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:35:54.686 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 podman[70552]: 2024-04-25 16:35:54.451639089 +0000 UTC m=+0.140252653 container create c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T16:35:54.686 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 podman[70552]: 2024-04-25 16:35:54.687727705 +0000 UTC m=+0.376341266 container init c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:35:55.095 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 podman[70552]: 2024-04-25 16:35:54.690511149 +0000 UTC m=+0.379124718 container start c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T16:35:55.096 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 ceph-mds[70573]: -- 172.21.15.112:0/1733859816 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5606e16e9a40 con 0x5606e1731600 2024-04-25T16:35:55.096 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j[70569]: starting mds.j at 2024-04-25T16:35:55.096 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 bash[70552]: c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 2024-04-25T16:35:55.096 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:35:54 smithi112 systemd[1]: Started Ceph mds.j for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1919525411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:55.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:55 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1919525411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:55.490 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:55 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1919525411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:55.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:55 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:56.410 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:56 smithi116 ceph-mon[32712]: Deploying daemon mds.d on smithi112 2024-04-25T16:35:56.410 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:56 smithi116 ceph-mon[32712]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:56.410 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:56 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.112:6834/759452882,v1:172.21.15.112:6835/759452882] up:boot 2024-04-25T16:35:56.410 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:56 smithi116 ceph-mon[32712]: fsmap 3 up:standby 2024-04-25T16:35:56.410 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:56 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T16:35:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:56 smithi133 ceph-mon[32691]: Deploying daemon mds.d on smithi112 2024-04-25T16:35:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:56 smithi133 ceph-mon[32691]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:56 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.112:6834/759452882,v1:172.21.15.112:6835/759452882] up:boot 2024-04-25T16:35:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:56 smithi133 ceph-mon[32691]: fsmap 3 up:standby 2024-04-25T16:35:56.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:56 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T16:35:56.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:56 smithi112 ceph-mon[29122]: Deploying daemon mds.d on smithi112 2024-04-25T16:35:56.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:56 smithi112 ceph-mon[29122]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:56.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:56 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.112:6834/759452882,v1:172.21.15.112:6835/759452882] up:boot 2024-04-25T16:35:56.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:56 smithi112 ceph-mon[29122]: fsmap 3 up:standby 2024-04-25T16:35:56.503 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:56 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T16:35:56.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 -- 172.21.15.112:0/2558175834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c108740 msgr2=0x7f492c10ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2558175834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c108740 0x7f492c10ab60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4920009860 tx=0x7f492002a340 comp rx=0 tx=0).stop 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 -- 172.21.15.112:0/2558175834 shutdown_connections 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2558175834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c108740 0x7f492c10ab60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2558175834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f492c103a40 0x7f492c103e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2558175834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c102800 0x7f492c103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 -- 172.21.15.112:0/2558175834 >> 172.21.15.112:0/2558175834 conn(0x7f492c0fd9f0 msgr2=0x7f492c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 -- 172.21.15.112:0/2558175834 shutdown_connections 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.657+0000 7f4931e9e640 1 -- 172.21.15.112:0/2558175834 wait complete. 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 Processor -- start 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 -- start start 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f492c102800 0x7f492c197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c103a40 0x7f492c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 0x7f492c19f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d220 con 0x7f492c108740 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d0a0 con 0x7f492c102800 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f4931e9e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d3a0 con 0x7f492c103a40 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f493169d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 0x7f492c19f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f493169d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 0x7f492c19f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36720/0 (socket says 172.21.15.112:36720) 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f493169d640 1 -- 172.21.15.112:0/2784105798 learned_addr learned my addr 172.21.15.112:0/2784105798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:56.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.658+0000 7f492bfff640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c103a40 0x7f492c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f4930e9c640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f492c102800 0x7f492c197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f493169d640 1 -- 172.21.15.112:0/2784105798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c103a40 msgr2=0x7f492c1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f493169d640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c103a40 0x7f492c1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f493169d640 1 -- 172.21.15.112:0/2784105798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f492c102800 msgr2=0x7f492c197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f493169d640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f492c102800 0x7f492c197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f493169d640 1 -- 172.21.15.112:0/2784105798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c19f8e0 con 0x7f492c108740 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f493169d640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 0x7f492c19f270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4920005940 tx=0x7f4920032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920040020 con 0x7f492c108740 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4920004040 con 0x7f492c108740 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c19fb70 con 0x7f492c108740 2024-04-25T16:35:56.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920037040 con 0x7f492c108740 2024-04-25T16:35:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.659+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c1a0060 con 0x7f492c108740 2024-04-25T16:35:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.661+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c107550 con 0x7f492c108740 2024-04-25T16:35:56.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.662+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f4920043020 con 0x7f492c108740 2024-04-25T16:35:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.663+0000 7f4929ffb640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f49140afae0 0x7f49140b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.663+0000 7f4930e9c640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f49140afae0 0x7f49140b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.663+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f49200fecc0 con 0x7f492c108740 2024-04-25T16:35:56.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.664+0000 7f4930e9c640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f49140afae0 0x7f49140b1ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f491c009530 tx=0x7f491c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:56.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.666+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f492004a020 con 0x7f492c108740 2024-04-25T16:35:56.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.899+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f492c02db50 con 0x7f492c108740 2024-04-25T16:35:56.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.900+0000 7f4929ffb640 1 -- 172.21.15.112:0/2784105798 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f49200c7650 con 0x7f492c108740 2024-04-25T16:35:56.899 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:56.899 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T16:35:56.899 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 4 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f49140afae0 msgr2=0x7f49140b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f49140afae0 0x7f49140b1ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f491c009530 tx=0x7f491c009260 comp rx=0 tx=0).stop 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 msgr2=0x7f492c19f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 0x7f492c19f270 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4920005940 tx=0x7f4920032dc0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 shutdown_connections 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f49140afae0 0x7f49140b1ff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f492c103a40 0x7f492c1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f492c108740 0x7f492c19f270 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 --2- 172.21.15.112:0/2784105798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f492c102800 0x7f492c197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 >> 172.21.15.112:0/2784105798 conn(0x7f492c0fd9f0 msgr2=0x7f492c104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 shutdown_connections 2024-04-25T16:35:56.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:56.902+0000 7f4931e9e640 1 -- 172.21.15.112:0/2784105798 wait complete. 2024-04-25T16:35:57.095 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:56 smithi112 systemd[1]: Starting Ceph mds.d for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:35:57.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:57 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2784105798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:57.458 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:57 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2784105798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:57.458 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 podman[70900]: 2024-04-25 16:35:57.095259093 +0000 UTC m=+0.017844479 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:35:57.458 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 podman[70900]: 2024-04-25 16:35:57.224536004 +0000 UTC m=+0.147121424 container create b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:35:57.458 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 podman[70900]: 2024-04-25 16:35:57.460561794 +0000 UTC m=+0.383147174 container init b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:35:57.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:57 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2784105798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:35:57.845 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 podman[70900]: 2024-04-25 16:35:57.463288322 +0000 UTC m=+0.385873700 container start b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T16:35:57.846 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 ceph-mds[70919]: -- 172.21.15.112:0/4197034691 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565cada3a40 con 0x5565cadeb180 2024-04-25T16:35:57.846 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d[70915]: starting mds.d at 2024-04-25T16:35:57.846 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 bash[70900]: b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 2024-04-25T16:35:57.846 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:35:57 smithi112 systemd[1]: Started Ceph mds.d for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:58.421 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:58 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:58.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:58.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:58.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:58 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:58.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:35:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:35:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:35:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:35:58.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:58 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:35:58.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:35:59.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4082082404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc108920 msgr2=0x7fa2bc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4082082404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc108920 0x7fa2bc10ad40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ca0 tx=0x7fa2b402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4082082404 shutdown_connections 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4082082404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc108920 0x7fa2bc10ad40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4082082404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2bc103c20 0x7fa2bc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4082082404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2bc1029e0 0x7fa2bc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4082082404 >> 172.21.15.112:0/4082082404 conn(0x7fa2bc0fdbf0 msgr2=0x7fa2bc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4082082404 shutdown_connections 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.039+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4082082404 wait complete. 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 Processor -- start 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 -- start start 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2bc1029e0 0x7fa2bc19a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 0x7fa2bc19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2bc108920 0x7fa2bc1a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc10f610 con 0x7fa2bc103c20 2024-04-25T16:35:59.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc10f490 con 0x7fa2bc108920 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c2f57640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc10f790 con 0x7fa2bc1029e0 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 0x7fa2bc19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 0x7fa2bc19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36770/0 (socket says 172.21.15.112:36770) 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2bbfff640 1 -- 172.21.15.112:0/4002712897 learned_addr learned my addr 172.21.15.112:0/4002712897 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c14f4640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2bc108920 0x7fa2bc1a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.040+0000 7fa2c0cf3640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2bc1029e0 0x7fa2bc19a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2bbfff640 1 -- 172.21.15.112:0/4002712897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2bc1029e0 msgr2=0x7fa2bc19a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2bbfff640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2bc1029e0 0x7fa2bc19a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2bbfff640 1 -- 172.21.15.112:0/4002712897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2bc108920 msgr2=0x7fa2bc1a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2bbfff640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2bc108920 0x7fa2bc1a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2bbfff640 1 -- 172.21.15.112:0/4002712897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc1a1d80 con 0x7fa2bc103c20 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2bbfff640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 0x7fa2bc19a780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800d950 tx=0x7fa2a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8017070 con 0x7fa2bc103c20 2024-04-25T16:35:59.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a8009c30 con 0x7fa2bc103c20 2024-04-25T16:35:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc1a2070 con 0x7fa2bc103c20 2024-04-25T16:35:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a800f620 con 0x7fa2bc103c20 2024-04-25T16:35:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.041+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1a2610 con 0x7fa2bc103c20 2024-04-25T16:35:59.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.042+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc107730 con 0x7fa2bc103c20 2024-04-25T16:35:59.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.043+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fa2a8010460 con 0x7fa2bc103c20 2024-04-25T16:35:59.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.044+0000 7fa2b9ffb640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2ac0afb70 0x7fa2ac0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:35:59.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.044+0000 7fa2c0cf3640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2ac0afb70 0x7fa2ac0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:35:59.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.045+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa2a80de2b0 con 0x7fa2bc103c20 2024-04-25T16:35:59.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.045+0000 7fa2c0cf3640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2ac0afb70 0x7fa2ac0b2080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005fd0 tx=0x7fa2b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:35:59.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.049+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2a8009de0 con 0x7fa2bc103c20 2024-04-25T16:35:59.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.272+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2bc02db50 con 0x7fa2bc103c20 2024-04-25T16:35:59.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.272+0000 7fa2b9ffb640 1 -- 172.21.15.112:0/4002712897 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7fa2a80a6ce0 con 0x7fa2bc103c20 2024-04-25T16:35:59.271 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:35:59.271 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T16:35:59.272 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 5 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2ac0afb70 msgr2=0x7fa2ac0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2ac0afb70 0x7fa2ac0b2080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0005fd0 tx=0x7fa2b000b390 comp rx=0 tx=0).stop 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 msgr2=0x7fa2bc19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 0x7fa2bc19a780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800d950 tx=0x7fa2a800de10 comp rx=0 tx=0).stop 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 shutdown_connections 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2bc108920 0x7fa2bc1a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa2bc103c20 0x7fa2bc19a780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2ac0afb70 0x7fa2ac0b2080 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 --2- 172.21.15.112:0/4002712897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2bc1029e0 0x7fa2bc19a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 >> 172.21.15.112:0/4002712897 conn(0x7fa2bc0fdbf0 msgr2=0x7fa2bc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 shutdown_connections 2024-04-25T16:35:59.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:35:59.275+0000 7fa2c2f57640 1 -- 172.21.15.112:0/4002712897 wait complete. 2024-04-25T16:35:59.284 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:59 smithi112 ceph-mon[29122]: Deploying daemon mds.e on smithi116 2024-04-25T16:35:59.284 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:59 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.112:6836/1910235790,v1:172.21.15.112:6837/1910235790] up:boot 2024-04-25T16:35:59.284 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:59 smithi112 ceph-mon[29122]: fsmap 4 up:standby 2024-04-25T16:35:59.284 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:35:59 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T16:35:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:59 smithi133 ceph-mon[32691]: Deploying daemon mds.e on smithi116 2024-04-25T16:35:59.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:59 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.112:6836/1910235790,v1:172.21.15.112:6837/1910235790] up:boot 2024-04-25T16:35:59.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:59 smithi133 ceph-mon[32691]: fsmap 4 up:standby 2024-04-25T16:35:59.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:35:59 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T16:35:59.477 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:59 smithi116 ceph-mon[32712]: Deploying daemon mds.e on smithi116 2024-04-25T16:35:59.477 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:59 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.112:6836/1910235790,v1:172.21.15.112:6837/1910235790] up:boot 2024-04-25T16:35:59.477 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:59 smithi116 ceph-mon[32712]: fsmap 4 up:standby 2024-04-25T16:35:59.477 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:35:59 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T16:36:00.168 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 podman[62743]: 2024-04-25 16:36:00.090301293 +0000 UTC m=+0.141911727 container create a56bba8a68dd3f6f886a37ec839c327810d4a79ea13c55c80b1088046dc8d2ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T16:36:00.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:00 smithi133 ceph-mon[32691]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:00.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:00 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4002712897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:00.457 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:00 smithi116 ceph-mon[32712]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:00.457 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:00 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4002712897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:00.458 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 podman[62743]: 2024-04-25 16:36:00.35971219 +0000 UTC m=+0.411322623 container init a56bba8a68dd3f6f886a37ec839c327810d4a79ea13c55c80b1088046dc8d2ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:36:00.458 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 podman[62743]: 2024-04-25 16:36:00.362236767 +0000 UTC m=+0.413847206 container start a56bba8a68dd3f6f886a37ec839c327810d4a79ea13c55c80b1088046dc8d2ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:36:00.458 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 ceph-mds[62762]: -- 172.21.15.116:0/3622681188 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55829408ba40 con 0x5582940d3a80 2024-04-25T16:36:00.458 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: starting mds.e at 2024-04-25T16:36:00.458 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 bash[62743]: a56bba8a68dd3f6f886a37ec839c327810d4a79ea13c55c80b1088046dc8d2ae 2024-04-25T16:36:00.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:00 smithi112 ceph-mon[29122]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:00.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:00 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4002712897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:00.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:36:00 smithi116 systemd[1]: Started Ceph mds.e for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:01.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:01.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.408+0000 7f01b270d640 1 -- 172.21.15.112:0/2826141058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1033c0 msgr2=0x7f01ac1037d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.408+0000 7f01b270d640 1 --2- 172.21.15.112:0/2826141058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1033c0 0x7f01ac1037d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009900 tx=0x7f01a002a410 comp rx=0 tx=0).stop 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 -- 172.21.15.112:0/2826141058 shutdown_connections 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 --2- 172.21.15.112:0/2826141058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac1080c0 0x7f01ac10a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 --2- 172.21.15.112:0/2826141058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1033c0 0x7f01ac1037d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 --2- 172.21.15.112:0/2826141058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac102180 0x7f01ac102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 -- 172.21.15.112:0/2826141058 >> 172.21.15.112:0/2826141058 conn(0x7f01ac0fd3b0 msgr2=0x7f01ac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 -- 172.21.15.112:0/2826141058 shutdown_connections 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 -- 172.21.15.112:0/2826141058 wait complete. 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 Processor -- start 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.409+0000 7f01b270d640 1 -- start start 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac102180 0x7f01ac197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:01.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac1033c0 0x7f01ac198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 0x7f01ac19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10d410 con 0x7f01ac1080c0 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10d290 con 0x7f01ac1033c0 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac10d590 con 0x7f01ac102180 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 0x7f01ac19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 0x7f01ac19f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36796/0 (socket says 172.21.15.112:36796) 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 -- 172.21.15.112:0/3599785612 learned_addr learned my addr 172.21.15.112:0/3599785612 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01abfff640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac102180 0x7f01ac197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01ab7fe640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac1033c0 0x7f01ac198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 -- 172.21.15.112:0/3599785612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac102180 msgr2=0x7f01ac197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac102180 0x7f01ac197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 -- 172.21.15.112:0/3599785612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac1033c0 msgr2=0x7f01ac198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac1033c0 0x7f01ac198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 -- 172.21.15.112:0/3599785612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac19fb00 con 0x7f01ac1080c0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01ab7fe640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac1033c0 0x7f01ac198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b0caa640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 0x7f01ac19f490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01a400eb10 tx=0x7f01a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4004b20 con 0x7f01ac1080c0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac19fdf0 con 0x7f01ac1080c0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a4004cc0 con 0x7f01ac1080c0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.410+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4010690 con 0x7f01ac1080c0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.411+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a0260 con 0x7f01ac1080c0 2024-04-25T16:36:01.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.412+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac106ed0 con 0x7f01ac1080c0 2024-04-25T16:36:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.413+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f01a4005000 con 0x7f01ac1080c0 2024-04-25T16:36:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.414+0000 7f01a97fa640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f01940afaf0 0x7f01940b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.414+0000 7f01abfff640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f01940afaf0 0x7f01940b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.414+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f01a40defa0 con 0x7f01ac1080c0 2024-04-25T16:36:01.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.414+0000 7f01abfff640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f01940afaf0 0x7f01940b2000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f01ac102d80 tx=0x7f019c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:01.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.417+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f01a4053ff0 con 0x7f01ac1080c0 2024-04-25T16:36:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:01.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: Deploying daemon mds.f on smithi133 2024-04-25T16:36:01.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:01.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6834/4202555064,v1:172.21.15.116:6835/4202555064] up:boot 2024-04-25T16:36:01.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: fsmap 5 up:standby 2024-04-25T16:36:01.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:01 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:36:01.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.638+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01ac02db50 con 0x7f01ac1080c0 2024-04-25T16:36:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.639+0000 7f01a97fa640 1 -- 172.21.15.112:0/3599785612 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7f01a40a8020 con 0x7f01ac1080c0 2024-04-25T16:36:01.638 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:01.638 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T16:36:01.638 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 6 2024-04-25T16:36:01.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f01940afaf0 msgr2=0x7f01940b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:01.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f01940afaf0 0x7f01940b2000 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f01ac102d80 tx=0x7f019c010040 comp rx=0 tx=0).stop 2024-04-25T16:36:01.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 msgr2=0x7f01ac19f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:01.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 0x7f01ac19f490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f01a400eb10 tx=0x7f01a400c2d0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 shutdown_connections 2024-04-25T16:36:01.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01ac1080c0 0x7f01ac19f490 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac1033c0 0x7f01ac198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f01940afaf0 0x7f01940b2000 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 --2- 172.21.15.112:0/3599785612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01ac102180 0x7f01ac197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:01.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 >> 172.21.15.112:0/3599785612 conn(0x7f01ac0fd3b0 msgr2=0x7f01ac1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:01.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 shutdown_connections 2024-04-25T16:36:01.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:01.641+0000 7f01b270d640 1 -- 172.21.15.112:0/3599785612 wait complete. 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: Deploying daemon mds.f on smithi133 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:01.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6834/4202555064,v1:172.21.15.116:6835/4202555064] up:boot 2024-04-25T16:36:01.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: fsmap 5 up:standby 2024-04-25T16:36:01.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:01 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: Deploying daemon mds.f on smithi133 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:02.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6834/4202555064,v1:172.21.15.116:6835/4202555064] up:boot 2024-04-25T16:36:02.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: fsmap 5 up:standby 2024-04-25T16:36:02.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:01 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:36:02.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:02 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3599785612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:02.895 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:02 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3599785612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:02.896 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:02 smithi133 systemd[1]: Starting Ceph mds.f for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:03.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:02 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3599785612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:03.166 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:02 smithi133 podman[63125]: 2024-04-25 16:36:02.799840316 +0000 UTC m=+0.017933457 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:03.167 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:02 smithi133 podman[63125]: 2024-04-25 16:36:02.905540788 +0000 UTC m=+0.123633946 container create b1e19a565213f4983fd75466ffc85ef414d91190612d10b938a48f45570905a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T16:36:03.167 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:03 smithi133 podman[63125]: 2024-04-25 16:36:03.132995455 +0000 UTC m=+0.351088568 container init b1e19a565213f4983fd75466ffc85ef414d91190612d10b938a48f45570905a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:36:03.167 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:03 smithi133 podman[63125]: 2024-04-25 16:36:03.13546946 +0000 UTC m=+0.353562572 container start b1e19a565213f4983fd75466ffc85ef414d91190612d10b938a48f45570905a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T16:36:03.167 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mds[63144]: -- 172.21.15.133:0/562500553 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5624199a5a40 con 0x5624199ed600 2024-04-25T16:36:03.449 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f[63140]: starting mds.f at 2024-04-25T16:36:03.450 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:03 smithi133 bash[63125]: b1e19a565213f4983fd75466ffc85ef414d91190612d10b938a48f45570905a3 2024-04-25T16:36:03.450 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:36:03 smithi133 systemd[1]: Started Ceph mds.f for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:03.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 -- 172.21.15.112:0/4191067777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 msgr2=0x7fad641054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 --2- 172.21.15.112:0/4191067777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641054b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fad54009900 tx=0x7fad5402a410 comp rx=0 tx=0).stop 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 -- 172.21.15.112:0/4191067777 shutdown_connections 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 --2- 172.21.15.112:0/4191067777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102250 0x7fad6410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 --2- 172.21.15.112:0/4191067777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad64101750 0x7fad64101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 --2- 172.21.15.112:0/4191067777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641054b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 -- 172.21.15.112:0/4191067777 >> 172.21.15.112:0/4191067777 conn(0x7fad640fd3b0 msgr2=0x7fad640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 -- 172.21.15.112:0/4191067777 shutdown_connections 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 -- 172.21.15.112:0/4191067777 wait complete. 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.763+0000 7fad69e20640 1 Processor -- start 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 -- start start 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad64101750 0x7fad6419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102250 0x7fad6419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410f610 con 0x7fad641050a0 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410f490 con 0x7fad64101750 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad6410f790 con 0x7fad64102250 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:03.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36806/0 (socket says 172.21.15.112:36806) 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 -- 172.21.15.112:0/3219103068 learned_addr learned my addr 172.21.15.112:0/3219103068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad62ffd640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102250 0x7fad6419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad637fe640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad64101750 0x7fad6419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 -- 172.21.15.112:0/3219103068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102250 msgr2=0x7fad6419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102250 0x7fad6419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 -- 172.21.15.112:0/3219103068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad64101750 msgr2=0x7fad6419a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad64101750 0x7fad6419a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 -- 172.21.15.112:0/3219103068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad641a1d40 con 0x7fad641050a0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad63fff640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641a16d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00eb10 tx=0x7fad5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c004b20 con 0x7fad641050a0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.764+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad641a1fd0 con 0x7fad641050a0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.765+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c004cc0 con 0x7fad641050a0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.765+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5c010690 con 0x7fad641050a0 2024-04-25T16:36:03.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.765+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad641a24e0 con 0x7fad641050a0 2024-04-25T16:36:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.765+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad64108a30 con 0x7fad641050a0 2024-04-25T16:36:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.767+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fad5c005000 con 0x7fad641050a0 2024-04-25T16:36:03.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.767+0000 7fad60ff9640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fad4c0afaf0 0x7fad4c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.768+0000 7fad637fe640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fad4c0afaf0 0x7fad4c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.768+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fad5c0deeb0 con 0x7fad641050a0 2024-04-25T16:36:03.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.768+0000 7fad637fe640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fad4c0afaf0 0x7fad4c0b2000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad54005ab0 tx=0x7fad54031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:03.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.771+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad5c053ff0 con 0x7fad641050a0 2024-04-25T16:36:03.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:03.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:03.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:03.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:03.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:03.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:03.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:03 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:03.938 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:03.938 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:03.938 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:03.938 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:03.938 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:03.939 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:03.939 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:03 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:03.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.988+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad6402db50 con 0x7fad641050a0 2024-04-25T16:36:03.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.988+0000 7fad60ff9640 1 -- 172.21.15.112:0/3219103068 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7fad5c0a7f30 con 0x7fad641050a0 2024-04-25T16:36:03.988 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:03.988 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T16:36:03.989 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-04-25T16:36:03.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fad4c0afaf0 msgr2=0x7fad4c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:03.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fad4c0afaf0 0x7fad4c0b2000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad54005ab0 tx=0x7fad54031400 comp rx=0 tx=0).stop 2024-04-25T16:36:03.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 msgr2=0x7fad641a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641a16d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad5c00eb10 tx=0x7fad5c00c2d0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 shutdown_connections 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fad4c0afaf0 0x7fad4c0b2000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad64102250 0x7fad6419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad641050a0 0x7fad641a16d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 --2- 172.21.15.112:0/3219103068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad64101750 0x7fad6419a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.990+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 >> 172.21.15.112:0/3219103068 conn(0x7fad640fd3b0 msgr2=0x7fad64105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.991+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 shutdown_connections 2024-04-25T16:36:03.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:03.991+0000 7fad69e20640 1 -- 172.21.15.112:0/3219103068 wait complete. 2024-04-25T16:36:04.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:04.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:04.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:04.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:04.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:04.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:04.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:03 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:04.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:04 smithi112 ceph-mon[29122]: Deploying daemon mds.c on smithi133 2024-04-25T16:36:04.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:04 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6834/2304762610,v1:172.21.15.133:6835/2304762610] up:boot 2024-04-25T16:36:04.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:04 smithi112 ceph-mon[29122]: fsmap 6 up:standby 2024-04-25T16:36:04.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:04 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:36:04.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:04 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3219103068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:04.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:04 smithi133 ceph-mon[32691]: Deploying daemon mds.c on smithi133 2024-04-25T16:36:04.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:04 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6834/2304762610,v1:172.21.15.133:6835/2304762610] up:boot 2024-04-25T16:36:04.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:04 smithi133 ceph-mon[32691]: fsmap 6 up:standby 2024-04-25T16:36:04.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:04 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:36:04.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:04 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3219103068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:05.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:04 smithi116 ceph-mon[32712]: Deploying daemon mds.c on smithi133 2024-04-25T16:36:05.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:04 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6834/2304762610,v1:172.21.15.133:6835/2304762610] up:boot 2024-04-25T16:36:05.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:04 smithi116 ceph-mon[32712]: fsmap 6 up:standby 2024-04-25T16:36:05.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:04 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:36:05.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:04 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3219103068' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:05.556 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 systemd[1]: Starting Ceph mds.c for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:05.809 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:05 smithi133 ceph-mon[32691]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:05.810 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 podman[63436]: 2024-04-25 16:36:05.569821516 +0000 UTC m=+0.019975229 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:05.810 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 podman[63436]: 2024-04-25 16:36:05.694046485 +0000 UTC m=+0.144200183 container create c5cc1be113a3f2a855ff95999ba2a59b729bccbf5580ef5a392d4ba9f80a9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T16:36:05.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:05 smithi112 ceph-mon[29122]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:06.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:06.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:05 smithi116 ceph-mon[32712]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 -- 172.21.15.112:0/3864727968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5378101750 msgr2=0x7f5378101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 --2- 172.21.15.112:0/3864727968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5378101750 0x7f5378101bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f536c009900 tx=0x7f536c02a410 comp rx=0 tx=0).stop 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 -- 172.21.15.112:0/3864727968 shutdown_connections 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 --2- 172.21.15.112:0/3864727968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5378102250 0x7f537810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 --2- 172.21.15.112:0/3864727968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5378101750 0x7f5378101bc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 --2- 172.21.15.112:0/3864727968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53781050a0 0x7f53781054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 -- 172.21.15.112:0/3864727968 >> 172.21.15.112:0/3864727968 conn(0x7f53780fd370 msgr2=0x7f53780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 -- 172.21.15.112:0/3864727968 shutdown_connections 2024-04-25T16:36:06.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.127+0000 7f537e72d640 1 -- 172.21.15.112:0/3864727968 wait complete. 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 Processor -- start 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 -- start start 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5378101750 0x7f537819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5378102250 0x7f537819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 0x7f53781a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f537810f630 con 0x7f53781050a0 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f537810f4b0 con 0x7f5378102250 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537e72d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f537810f7b0 con 0x7f5378101750 2024-04-25T16:36:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537ccca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 0x7f53781a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537ccca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 0x7f53781a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41700/0 (socket says 172.21.15.112:41700) 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f537ccca640 1 -- 172.21.15.112:0/2507313462 learned_addr learned my addr 172.21.15.112:0/2507313462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f53777fe640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5378102250 0x7f537819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.128+0000 7f5377fff640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5378101750 0x7f537819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537ccca640 1 -- 172.21.15.112:0/2507313462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5378101750 msgr2=0x7f537819a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537ccca640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5378101750 0x7f537819a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537ccca640 1 -- 172.21.15.112:0/2507313462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5378102250 msgr2=0x7f537819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537ccca640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5378102250 0x7f537819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537ccca640 1 -- 172.21.15.112:0/2507313462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53781a1d20 con 0x7f53781050a0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537ccca640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 0x7f53781a16b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f537000eb10 tx=0x7f537000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370004b20 con 0x7f53781050a0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5370004cc0 con 0x7f53781050a0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53781a1fb0 con 0x7f53781050a0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5370010670 con 0x7f53781050a0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.129+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53781a24c0 con 0x7f53781050a0 2024-04-25T16:36:06.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.130+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5378108a30 con 0x7f53781050a0 2024-04-25T16:36:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.131+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f5370005000 con 0x7f53781050a0 2024-04-25T16:36:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.132+0000 7f53757fa640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f53600afb70 0x7f53600b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.132+0000 7f5377fff640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f53600afb70 0x7f53600b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.132+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f53700dee80 con 0x7f53781050a0 2024-04-25T16:36:06.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.132+0000 7f5377fff640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f53600afb70 0x7f53600b2080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f537806b320 tx=0x7f536800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:06.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.136+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5370053ff0 con 0x7f53781050a0 2024-04-25T16:36:06.199 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 podman[63436]: 2024-04-25 16:36:05.930536005 +0000 UTC m=+0.380689702 container init c5cc1be113a3f2a855ff95999ba2a59b729bccbf5580ef5a392d4ba9f80a9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:36:06.200 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 podman[63436]: 2024-04-25 16:36:05.933343338 +0000 UTC m=+0.383497037 container start c5cc1be113a3f2a855ff95999ba2a59b729bccbf5580ef5a392d4ba9f80a9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T16:36:06.200 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 ceph-mds[63455]: -- 172.21.15.133:0/3227872489 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cae9aa3a40 con 0x55cae9aeb180 2024-04-25T16:36:06.200 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c[63451]: starting mds.c at 2024-04-25T16:36:06.200 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:06 smithi133 bash[63436]: c5cc1be113a3f2a855ff95999ba2a59b729bccbf5580ef5a392d4ba9f80a9987 2024-04-25T16:36:06.200 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:36:06 smithi133 systemd[1]: Started Ceph mds.c for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.351+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f537802db50 con 0x7f53781050a0 2024-04-25T16:36:06.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.352+0000 7f53757fa640 1 -- 172.21.15.112:0/2507313462 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7f53700a7f00 con 0x7f53781050a0 2024-04-25T16:36:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:06.351 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T16:36:06.351 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 7 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f53600afb70 msgr2=0x7f53600b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f53600afb70 0x7f53600b2080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f537806b320 tx=0x7f536800b390 comp rx=0 tx=0).stop 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 msgr2=0x7f53781a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 0x7f53781a16b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f537000eb10 tx=0x7f537000c2d0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 shutdown_connections 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53781050a0 0x7f53781a16b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5378102250 0x7f537819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f53600afb70 0x7f53600b2080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 --2- 172.21.15.112:0/2507313462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5378101750 0x7f537819a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 >> 172.21.15.112:0/2507313462 conn(0x7f53780fd370 msgr2=0x7f5378105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 shutdown_connections 2024-04-25T16:36:06.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:06.354+0000 7f537e72d640 1 -- 172.21.15.112:0/2507313462 wait complete. 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:07.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: Deploying daemon mds.h on smithi116 2024-04-25T16:36:07.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2507313462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:07.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6836/3840802165,v1:172.21.15.133:6837/3840802165] up:boot 2024-04-25T16:36:07.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: fsmap 7 up:standby 2024-04-25T16:36:07.260 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:07 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: Deploying daemon mds.h on smithi116 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2507313462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6836/3840802165,v1:172.21.15.133:6837/3840802165] up:boot 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: fsmap 7 up:standby 2024-04-25T16:36:07.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:07 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: Deploying daemon mds.h on smithi116 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2507313462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6836/3840802165,v1:172.21.15.133:6837/3840802165] up:boot 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: fsmap 7 up:standby 2024-04-25T16:36:07.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:07 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:36:08.333 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:08 smithi116 ceph-mon[32712]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:08.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:08 smithi112 ceph-mon[29122]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:08.364 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:08.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:08 smithi133 ceph-mon[32691]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 -- 172.21.15.112:0/2101810120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1029c0 msgr2=0x7f7d8c1036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 --2- 172.21.15.112:0/2101810120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1029c0 0x7f7d8c1036c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d84009ca0 tx=0x7f7d8402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 -- 172.21.15.112:0/2101810120 shutdown_connections 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 --2- 172.21.15.112:0/2101810120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8c103c00 0x7f7d8c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 --2- 172.21.15.112:0/2101810120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c108900 0x7f7d8c10ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 --2- 172.21.15.112:0/2101810120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c1029c0 0x7f7d8c1036c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 -- 172.21.15.112:0/2101810120 >> 172.21.15.112:0/2101810120 conn(0x7f7d8c0fdbd0 msgr2=0x7f7d8c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 -- 172.21.15.112:0/2101810120 shutdown_connections 2024-04-25T16:36:08.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.487+0000 7f7d9340d640 1 -- 172.21.15.112:0/2101810120 wait complete. 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 Processor -- start 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 -- start start 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8c1029c0 0x7f7d8c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c103c00 0x7f7d8c19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 0x7f7d8c1a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c10e4c0 con 0x7f7d8c108900 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c10e340 con 0x7f7d8c103c00 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d9340d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c10e640 con 0x7f7d8c1029c0 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 0x7f7d8c1a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:08.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 0x7f7d8c1a1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41720/0 (socket says 172.21.15.112:41720) 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 -- 172.21.15.112:0/1344634429 learned_addr learned my addr 172.21.15.112:0/1344634429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d909a8640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c103c00 0x7f7d8c19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d911a9640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8c1029c0 0x7f7d8c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 -- 172.21.15.112:0/1344634429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8c1029c0 msgr2=0x7f7d8c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8c1029c0 0x7f7d8c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 -- 172.21.15.112:0/1344634429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c103c00 msgr2=0x7f7d8c19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c103c00 0x7f7d8c19a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 -- 172.21.15.112:0/1344634429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c1a1da0 con 0x7f7d8c108900 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d909a8640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c103c00 0x7f7d8c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.488+0000 7f7d919aa640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 0x7f7d8c1a1730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e9b0 tx=0x7f7d7c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.489+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00cd80 con 0x7f7d8c108900 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.489+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d7c0035f0 con 0x7f7d8c108900 2024-04-25T16:36:08.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.489+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c1a2090 con 0x7f7d8c108900 2024-04-25T16:36:08.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.489+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c010690 con 0x7f7d8c108900 2024-04-25T16:36:08.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.489+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c1a2590 con 0x7f7d8c108900 2024-04-25T16:36:08.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.490+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c1a2850 con 0x7f7d8c108900 2024-04-25T16:36:08.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.491+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f7d7c004000 con 0x7f7d8c108900 2024-04-25T16:36:08.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.492+0000 7f7d8a7fc640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7d740afaf0 0x7f7d740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:08.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.492+0000 7f7d911a9640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7d740afaf0 0x7f7d740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:08.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.492+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7d7c0dec30 con 0x7f7d8c108900 2024-04-25T16:36:08.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.492+0000 7f7d911a9640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7d740afaf0 0x7f7d740b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d84000f80 tx=0x7f7d84004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:08.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.496+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d7c0a7d40 con 0x7f7d8c108900 2024-04-25T16:36:08.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.709+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d8c1a2a20 con 0x7f7d8c108900 2024-04-25T16:36:08.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.709+0000 7f7d8a7fc640 1 -- 172.21.15.112:0/1344634429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7f7d7c0a7640 con 0x7f7d8c108900 2024-04-25T16:36:08.708 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:08.708 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T16:36:08.709 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T16:36:08.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7d740afaf0 msgr2=0x7f7d740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:08.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7d740afaf0 0x7f7d740b2000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d84000f80 tx=0x7f7d84004a40 comp rx=0 tx=0).stop 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 msgr2=0x7f7d8c1a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 0x7f7d8c1a1730 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e9b0 tx=0x7f7d7c00ee70 comp rx=0 tx=0).stop 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 shutdown_connections 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d8c108900 0x7f7d8c1a1730 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d8c103c00 0x7f7d8c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7d740afaf0 0x7f7d740b2000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 --2- 172.21.15.112:0/1344634429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d8c1029c0 0x7f7d8c19a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.711+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 >> 172.21.15.112:0/1344634429 conn(0x7f7d8c0fdbd0 msgr2=0x7f7d8c109e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.712+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 shutdown_connections 2024-04-25T16:36:08.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:08.712+0000 7f7d9340d640 1 -- 172.21.15.112:0/1344634429 wait complete. 2024-04-25T16:36:08.728 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:08 smithi116 systemd[1]: Starting Ceph mds.h for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:09.008 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:08 smithi116 podman[63054]: 2024-04-25 16:36:08.633255716 +0000 UTC m=+0.017903886 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:09.009 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:08 smithi116 podman[63054]: 2024-04-25 16:36:08.763092212 +0000 UTC m=+0.147740347 container create bd9c689a7e37b2e59adea4e63ec3f3e46492455d21a6d40cf4dcb750c96e09f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:36:09.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:09 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1344634429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:09 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1344634429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:09.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:09 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1344634429' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:09.509 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:09 smithi116 podman[63054]: 2024-04-25 16:36:09.043814664 +0000 UTC m=+0.428462798 container init bd9c689a7e37b2e59adea4e63ec3f3e46492455d21a6d40cf4dcb750c96e09f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T16:36:09.509 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:09 smithi116 podman[63054]: 2024-04-25 16:36:09.046474087 +0000 UTC m=+0.431122235 container start bd9c689a7e37b2e59adea4e63ec3f3e46492455d21a6d40cf4dcb750c96e09f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T16:36:09.509 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:09 smithi116 ceph-mds[63073]: -- 172.21.15.116:0/2204035027 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55828d00da40 con 0x55828d055180 2024-04-25T16:36:09.509 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h[63069]: starting mds.h at 2024-04-25T16:36:09.509 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:09 smithi116 bash[63054]: bd9c689a7e37b2e59adea4e63ec3f3e46492455d21a6d40cf4dcb750c96e09f6 2024-04-25T16:36:09.509 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:36:09 smithi116 systemd[1]: Started Ceph mds.h for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: Deploying daemon mds.i on smithi133 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] up:boot 2024-04-25T16:36:10.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: fsmap 8 up:standby 2024-04-25T16:36:10.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:10 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:36:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: Deploying daemon mds.i on smithi133 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] up:boot 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: fsmap 8 up:standby 2024-04-25T16:36:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:10 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: Deploying daemon mds.i on smithi133 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] up:boot 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: fsmap 8 up:standby 2024-04-25T16:36:10.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:10 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:36:10.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 -- 172.21.15.112:0/661381555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718102a00 msgr2=0x7f2718103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 --2- 172.21.15.112:0/661381555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718102a00 0x7f2718103700 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f270c009900 tx=0x7f270c02a410 comp rx=0 tx=0).stop 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 -- 172.21.15.112:0/661381555 shutdown_connections 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 --2- 172.21.15.112:0/661381555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718108940 0x7f271810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 --2- 172.21.15.112:0/661381555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 0x7f2718104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 --2- 172.21.15.112:0/661381555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718102a00 0x7f2718103700 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 -- 172.21.15.112:0/661381555 >> 172.21.15.112:0/661381555 conn(0x7f27180fdbf0 msgr2=0x7f27181000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:10.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 -- 172.21.15.112:0/661381555 shutdown_connections 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.844+0000 7f271e442640 1 -- 172.21.15.112:0/661381555 wait complete. 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 Processor -- start 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 -- start start 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 0x7f271810f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 0x7f271810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 0x7f271810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27180759e0 con 0x7f2718108940 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2718075860 con 0x7f2718103c40 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271e442640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2718075b60 con 0x7f2718102a00 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271c9df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 0x7f271810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f2717fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 0x7f271810f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:10.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f2717fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 0x7f271810f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:34784/0 (socket says 172.21.15.112:34784) 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f2717fff640 1 -- 172.21.15.112:0/577412120 learned_addr learned my addr 172.21.15.112:0/577412120 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f271c9df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 0x7f271810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41732/0 (socket says 172.21.15.112:41732) 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.845+0000 7f27177fe640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 0x7f271810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f2717fff640 1 -- 172.21.15.112:0/577412120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 msgr2=0x7f271810da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f2717fff640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 0x7f271810da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f2717fff640 1 -- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 msgr2=0x7f271810df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f2717fff640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 0x7f271810df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f2717fff640 1 -- 172.21.15.112:0/577412120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271819e030 con 0x7f2718102a00 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f27177fe640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 0x7f271810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f271c9df640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 0x7f271810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f2717fff640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 0x7f271810f300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f270c02a930 tx=0x7f270c031a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c03f030 con 0x7f2718102a00 2024-04-25T16:36:10.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271810e490 con 0x7f2718102a00 2024-04-25T16:36:10.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f270c003aa0 con 0x7f2718102a00 2024-04-25T16:36:10.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270c035070 con 0x7f2718102a00 2024-04-25T16:36:10.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.846+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181a8b40 con 0x7f2718102a00 2024-04-25T16:36:10.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.847+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2718107750 con 0x7f2718102a00 2024-04-25T16:36:10.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.849+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f270c042030 con 0x7f2718102a00 2024-04-25T16:36:10.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.849+0000 7f27157fa640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f27000afaf0 0x7f27000b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:10.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.849+0000 7f27177fe640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f27000afaf0 0x7f27000b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:10.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.849+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f270c0ff1d0 con 0x7f2718102a00 2024-04-25T16:36:10.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.850+0000 7f27177fe640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f27000afaf0 0x7f27000b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2708005fd0 tx=0x7f2708005f40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:10.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:10.853+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f270c0c8230 con 0x7f2718102a00 2024-04-25T16:36:11.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.068+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f271802db50 con 0x7f2718102a00 2024-04-25T16:36:11.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.069+0000 7f27157fa640 1 -- 172.21.15.112:0/577412120 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7186 (secure 0 0 0) 0x7f270c0c7b30 con 0x7f2718102a00 2024-04-25T16:36:11.068 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:11.068 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T16:36:11.068 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 9 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f27000afaf0 msgr2=0x7f27000b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f27000afaf0 0x7f27000b2000 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2708005fd0 tx=0x7f2708005f40 comp rx=0 tx=0).stop 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 msgr2=0x7f271810f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 0x7f271810f300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f270c02a930 tx=0x7f270c031a50 comp rx=0 tx=0).stop 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 shutdown_connections 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2718108940 0x7f271810df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2718103c40 0x7f271810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f27000afaf0 0x7f27000b2000 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 --2- 172.21.15.112:0/577412120 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2718102a00 0x7f271810f300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:11.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 >> 172.21.15.112:0/577412120 conn(0x7f27180fdbf0 msgr2=0x7f2718104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:11.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 shutdown_connections 2024-04-25T16:36:11.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:11.071+0000 7f271e442640 1 -- 172.21.15.112:0/577412120 wait complete. 2024-04-25T16:36:11.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:11 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/577412120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:11.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:11 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/577412120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:11.450 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 systemd[1]: Starting Ceph mds.i for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:11.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:11 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/577412120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:11.800 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 podman[63744]: 2024-04-25 16:36:11.41223871 +0000 UTC m=+0.018312522 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:11.800 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 podman[63744]: 2024-04-25 16:36:11.549190742 +0000 UTC m=+0.155264543 container create a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T16:36:11.800 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 podman[63744]: 2024-04-25 16:36:11.80215855 +0000 UTC m=+0.408232352 container init a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T16:36:12.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 podman[63744]: 2024-04-25 16:36:11.804878759 +0000 UTC m=+0.410952573 container start a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:36:12.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 ceph-mds[63763]: -- 172.21.15.133:0/1170914735 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b8723bfa40 con 0x55b872407600 2024-04-25T16:36:12.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i[63759]: starting mds.i at 2024-04-25T16:36:12.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 bash[63744]: a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 2024-04-25T16:36:12.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:36:11 smithi133 systemd[1]: Started Ceph mds.i for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:12.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:12.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:12 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:12.586 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:12 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:12.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:12.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:12.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:12.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:12.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:12 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:13.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 -- 172.21.15.112:0/2060494157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c109a50 msgr2=0x7fe99c10be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/2060494157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c109a50 0x7fe99c10be70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe994009ca0 tx=0x7fe99402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 -- 172.21.15.112:0/2060494157 shutdown_connections 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/2060494157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c109a50 0x7fe99c10be70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/2060494157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c104d50 0x7fe99c105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/2060494157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe99c103b10 0x7fe99c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 -- 172.21.15.112:0/2060494157 >> 172.21.15.112:0/2060494157 conn(0x7fe99c0fed40 msgr2=0x7fe99c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 -- 172.21.15.112:0/2060494157 shutdown_connections 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.195+0000 7fe9a2b00640 1 -- 172.21.15.112:0/2060494157 wait complete. 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 Processor -- start 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 -- start start 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe99c103b10 0x7fe99c19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 0x7fe99c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:13.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c109a50 0x7fe99c1a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10f640 con 0x7fe99c104d50 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10f4c0 con 0x7fe99c109a50 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a2b00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10f7c0 con 0x7fe99c103b10 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe99bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 0x7fe99c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe99bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 0x7fe99c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41754/0 (socket says 172.21.15.112:41754) 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe99bfff640 1 -- 172.21.15.112:0/148841752 learned_addr learned my addr 172.21.15.112:0/148841752 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a089c640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe99c103b10 0x7fe99c19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.196+0000 7fe9a109d640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c109a50 0x7fe99c1a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe99bfff640 1 -- 172.21.15.112:0/148841752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe99c103b10 msgr2=0x7fe99c19a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe99bfff640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe99c103b10 0x7fe99c19a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe99bfff640 1 -- 172.21.15.112:0/148841752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c109a50 msgr2=0x7fe99c1a16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe99bfff640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c109a50 0x7fe99c1a16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe99bfff640 1 -- 172.21.15.112:0/148841752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c1a1d50 con 0x7fe99c104d50 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe9a109d640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c109a50 0x7fe99c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe99bfff640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 0x7fe99c19a750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe98800d950 tx=0x7fe98800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988017070 con 0x7fe99c104d50 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe988009c30 con 0x7fe99c104d50 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800f620 con 0x7fe99c104d50 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c1a1fe0 con 0x7fe99c104d50 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.197+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c1a2520 con 0x7fe99c104d50 2024-04-25T16:36:13.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.198+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99c108860 con 0x7fe99c104d50 2024-04-25T16:36:13.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.199+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fe988010460 con 0x7fe99c104d50 2024-04-25T16:36:13.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.200+0000 7fe999ffb640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe98c0afb70 0x7fe98c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:13.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.200+0000 7fe9a089c640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe98c0afb70 0x7fe98c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:13.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.200+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fe9880de360 con 0x7fe99c104d50 2024-04-25T16:36:13.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.200+0000 7fe9a089c640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe98c0afb70 0x7fe98c0b2080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe990005fd0 tx=0x7fe99000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:13.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.205+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9880a73e0 con 0x7fe99c104d50 2024-04-25T16:36:13.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.424+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe99c02db50 con 0x7fe99c104d50 2024-04-25T16:36:13.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.425+0000 7fe999ffb640 1 -- 172.21.15.112:0/148841752 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8010 (secure 0 0 0) 0x7fe988009de0 con 0x7fe99c104d50 2024-04-25T16:36:13.424 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:13.424 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T16:36:13.424 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 10 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe98c0afb70 msgr2=0x7fe98c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe98c0afb70 0x7fe98c0b2080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe990005fd0 tx=0x7fe99000b390 comp rx=0 tx=0).stop 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 msgr2=0x7fe99c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 0x7fe99c19a750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe98800d950 tx=0x7fe98800de10 comp rx=0 tx=0).stop 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 shutdown_connections 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe99c109a50 0x7fe99c1a16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe99c104d50 0x7fe99c19a750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe98c0afb70 0x7fe98c0b2080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 --2- 172.21.15.112:0/148841752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe99c103b10 0x7fe99c19a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 >> 172.21.15.112:0/148841752 conn(0x7fe99c0fed40 msgr2=0x7fe99c105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 shutdown_connections 2024-04-25T16:36:13.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:13.427+0000 7fe9a2b00640 1 -- 172.21.15.112:0/148841752 wait complete. 2024-04-25T16:36:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:13 smithi116 ceph-mon[32712]: Deploying daemon mds.g on smithi112 2024-04-25T16:36:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:13 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] up:boot 2024-04-25T16:36:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:13 smithi116 ceph-mon[32712]: fsmap 9 up:standby 2024-04-25T16:36:13.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:13 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:36:13.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:13 smithi112 ceph-mon[29122]: Deploying daemon mds.g on smithi112 2024-04-25T16:36:13.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:13 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] up:boot 2024-04-25T16:36:13.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:13 smithi112 ceph-mon[29122]: fsmap 9 up:standby 2024-04-25T16:36:13.584 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:13 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:36:13.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:13 smithi133 ceph-mon[32691]: Deploying daemon mds.g on smithi112 2024-04-25T16:36:13.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:13 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] up:boot 2024-04-25T16:36:13.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:13 smithi133 ceph-mon[32691]: fsmap 9 up:standby 2024-04-25T16:36:13.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:13 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:36:14.211 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 systemd[1]: Starting Ceph mds.g for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:14.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:14 smithi112 ceph-mon[29122]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:14.497 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:14 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/148841752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:14.498 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 podman[71504]: 2024-04-25 16:36:14.151782236 +0000 UTC m=+0.017518724 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:14.498 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 podman[71504]: 2024-04-25 16:36:14.263459237 +0000 UTC m=+0.129195703 container create 0361e0ebd79fe031babb65cf26cf1704084ff362b51a65c7124914c8b75e58fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T16:36:14.498 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 podman[71504]: 2024-04-25 16:36:14.499578359 +0000 UTC m=+0.365314823 container init 0361e0ebd79fe031babb65cf26cf1704084ff362b51a65c7124914c8b75e58fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T16:36:14.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:14 smithi116 ceph-mon[32712]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:14.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:14 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/148841752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:14.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:14 smithi133 ceph-mon[32691]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:14.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:14 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/148841752' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:14.845 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 podman[71504]: 2024-04-25 16:36:14.502284708 +0000 UTC m=+0.368021173 container start 0361e0ebd79fe031babb65cf26cf1704084ff362b51a65c7124914c8b75e58fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T16:36:14.845 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 ceph-mds[71523]: -- 172.21.15.112:0/749703267 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d504443a40 con 0x55d50448ba80 2024-04-25T16:36:14.846 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g[71519]: starting mds.g at 2024-04-25T16:36:14.846 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 bash[71504]: 0361e0ebd79fe031babb65cf26cf1704084ff362b51a65c7124914c8b75e58fc 2024-04-25T16:36:14.846 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:36:14 smithi112 systemd[1]: Started Ceph mds.g for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:15.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 -- 172.21.15.112:0/720499999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1050a0 msgr2=0x7fa22c1054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:15.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 --2- 172.21.15.112:0/720499999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1050a0 0x7fa22c1054b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009900 tx=0x7fa21c02a410 comp rx=0 tx=0).stop 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 -- 172.21.15.112:0/720499999 shutdown_connections 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 --2- 172.21.15.112:0/720499999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c101750 0x7fa22c101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 --2- 172.21.15.112:0/720499999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c102250 0x7fa22c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 --2- 172.21.15.112:0/720499999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c1050a0 0x7fa22c1054b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 -- 172.21.15.112:0/720499999 >> 172.21.15.112:0/720499999 conn(0x7fa22c0fd3b0 msgr2=0x7fa22c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 -- 172.21.15.112:0/720499999 shutdown_connections 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 -- 172.21.15.112:0/720499999 wait complete. 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.551+0000 7fa231e20640 1 Processor -- start 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 -- start start 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c101750 0x7fa22c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 0x7fa22c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c1050a0 0x7fa22c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:15.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10f660 con 0x7fa22c102250 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10f4e0 con 0x7fa22c1050a0 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa231e20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10f7e0 con 0x7fa22c101750 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 0x7fa22c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 0x7fa22c19a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36196/0 (socket says 172.21.15.112:36196) 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 -- 172.21.15.112:0/660225834 learned_addr learned my addr 172.21.15.112:0/660225834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22b7fe640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c101750 0x7fa22c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22bfff640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c1050a0 0x7fa22c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 -- 172.21.15.112:0/660225834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c101750 msgr2=0x7fa22c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c101750 0x7fa22c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 -- 172.21.15.112:0/660225834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c1050a0 msgr2=0x7fa22c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c1050a0 0x7fa22c1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 -- 172.21.15.112:0/660225834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c1a1d90 con 0x7fa22c102250 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.552+0000 7fa22affd640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 0x7fa22c19a7e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa22000b4a0 tx=0x7fa22000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.553+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220014070 con 0x7fa22c102250 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.553+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa220010ac0 con 0x7fa22c102250 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.553+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c1a2080 con 0x7fa22c102250 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.553+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa220017c30 con 0x7fa22c102250 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.553+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c1a2600 con 0x7fa22c102250 2024-04-25T16:36:15.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.554+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c108a30 con 0x7fa22c102250 2024-04-25T16:36:15.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.555+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fa220017070 con 0x7fa22c102250 2024-04-25T16:36:15.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.556+0000 7fa228ff9640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2140afaf0 0x7fa2140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:15.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.556+0000 7fa22b7fe640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2140afaf0 0x7fa2140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:15.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.556+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fa2200ddfe0 con 0x7fa22c102250 2024-04-25T16:36:15.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.556+0000 7fa22b7fe640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2140afaf0 0x7fa2140b2000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009230 tx=0x7fa21c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:15.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.559+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa2200a7060 con 0x7fa22c102250 2024-04-25T16:36:15.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.783+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa22c02db50 con 0x7fa22c102250 2024-04-25T16:36:15.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.784+0000 7fa228ff9640 1 -- 172.21.15.112:0/660225834 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7fa2200af0e0 con 0x7fa22c102250 2024-04-25T16:36:15.782 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:15.783 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T16:36:15.783 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 11 2024-04-25T16:36:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.785+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2140afaf0 msgr2=0x7fa2140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:15.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.785+0000 7fa231e20640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2140afaf0 0x7fa2140b2000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa21c009230 tx=0x7fa21c031400 comp rx=0 tx=0).stop 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 msgr2=0x7fa22c19a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 0x7fa22c19a7e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa22000b4a0 tx=0x7fa22000b960 comp rx=0 tx=0).stop 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 shutdown_connections 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c1050a0 0x7fa22c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa22c102250 0x7fa22c19a7e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa2140afaf0 0x7fa2140b2000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 --2- 172.21.15.112:0/660225834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa22c101750 0x7fa22c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 >> 172.21.15.112:0/660225834 conn(0x7fa22c0fd3b0 msgr2=0x7fa22c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 shutdown_connections 2024-04-25T16:36:15.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:15.786+0000 7fa231e20640 1 -- 172.21.15.112:0/660225834 wait complete. 2024-04-25T16:36:15.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: Deploying daemon mds.a on smithi112 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.112:6838/72433575,v1:172.21.15.112:6839/72433575] up:boot 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: fsmap 10 up:standby 2024-04-25T16:36:15.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: Deploying daemon mds.a on smithi112 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.112:6838/72433575,v1:172.21.15.112:6839/72433575] up:boot 2024-04-25T16:36:15.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: fsmap 10 up:standby 2024-04-25T16:36:15.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T16:36:16.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:16.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: Deploying daemon mds.a on smithi112 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.112:6838/72433575,v1:172.21.15.112:6839/72433575] up:boot 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: fsmap 10 up:standby 2024-04-25T16:36:16.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T16:36:16.910 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:16 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/660225834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:16.910 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:16 smithi112 systemd[1]: Starting Ceph mds.a for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:17.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:16 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/660225834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:17.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:16 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/660225834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:17.207 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 podman[71852]: 2024-04-25 16:36:16.978339476 +0000 UTC m=+0.017936041 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:17.207 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 podman[71852]: 2024-04-25 16:36:17.085634863 +0000 UTC m=+0.125231453 container create 8f1f0b22835a46ff01a867c006d1a08c79fb8e7e719b30397af986f2837fee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T16:36:17.596 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 podman[71852]: 2024-04-25 16:36:17.33865522 +0000 UTC m=+0.378251771 container init 8f1f0b22835a46ff01a867c006d1a08c79fb8e7e719b30397af986f2837fee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:36:17.596 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 podman[71852]: 2024-04-25 16:36:17.341254465 +0000 UTC m=+0.380851017 container start 8f1f0b22835a46ff01a867c006d1a08c79fb8e7e719b30397af986f2837fee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:36:17.596 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mds[71881]: -- 172.21.15.112:0/1291202854 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607c262da40 con 0x5607c2675a80 2024-04-25T16:36:17.596 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a[71875]: starting mds.a at 2024-04-25T16:36:17.596 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 bash[71852]: 8f1f0b22835a46ff01a867c006d1a08c79fb8e7e719b30397af986f2837fee01 2024-04-25T16:36:17.596 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:36:17 smithi112 systemd[1]: Started Ceph mds.a for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:17.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 -- 172.21.15.112:0/2749081279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981029c0 msgr2=0x7f3f981036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 --2- 172.21.15.112:0/2749081279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981029c0 0x7f3f981036c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009ca0 tx=0x7f3f9002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 -- 172.21.15.112:0/2749081279 shutdown_connections 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 --2- 172.21.15.112:0/2749081279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f98108900 0x7f3f9810ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 --2- 172.21.15.112:0/2749081279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f98103c00 0x7f3f98104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 --2- 172.21.15.112:0/2749081279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f981029c0 0x7f3f981036c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 -- 172.21.15.112:0/2749081279 >> 172.21.15.112:0/2749081279 conn(0x7f3f980fdbd0 msgr2=0x7f3f98100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 -- 172.21.15.112:0/2749081279 shutdown_connections 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.911+0000 7f3f9f277640 1 -- 172.21.15.112:0/2749081279 wait complete. 2024-04-25T16:36:17.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 Processor -- start 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 -- start start 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f981029c0 0x7f3f9819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f98103c00 0x7f3f9819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 0x7f3f981a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9810e4c0 con 0x7f3f98108900 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9810e340 con 0x7f3f981029c0 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9f277640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f9810e640 con 0x7f3f98103c00 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 0x7f3f981a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 0x7f3f981a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36224/0 (socket says 172.21.15.112:36224) 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 -- 172.21.15.112:0/627737536 learned_addr learned my addr 172.21.15.112:0/627737536 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d013640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f981029c0 0x7f3f9819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 -- 172.21.15.112:0/627737536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f98103c00 msgr2=0x7f3f9819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f98103c00 0x7f3f9819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 -- 172.21.15.112:0/627737536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f981029c0 msgr2=0x7f3f9819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:17.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f981029c0 0x7f3f9819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 -- 172.21.15.112:0/627737536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981a1d60 con 0x7f3f98108900 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.912+0000 7f3f9d814640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 0x7f3f981a16f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400d750 tx=0x7f3f9400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.913+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94017070 con 0x7f3f98108900 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.913+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f940054d0 con 0x7f3f98108900 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.913+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981a1ff0 con 0x7f3f98108900 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.913+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f94004c60 con 0x7f3f98108900 2024-04-25T16:36:17.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.913+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981a2510 con 0x7f3f98108900 2024-04-25T16:36:17.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.914+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f981a2740 con 0x7f3f98108900 2024-04-25T16:36:17.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.915+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f3f94009bd0 con 0x7f3f98108900 2024-04-25T16:36:17.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.916+0000 7f3f8e7fc640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3f840afaf0 0x7f3f840b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:17.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.916+0000 7f3f9d013640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3f840afaf0 0x7f3f840b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:17.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.916+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f3f940de110 con 0x7f3f98108900 2024-04-25T16:36:17.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.916+0000 7f3f9d013640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3f840afaf0 0x7f3f840b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f90000f80 tx=0x7f3f90004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:17.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:17.920+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3f940a7190 con 0x7f3f98108900 2024-04-25T16:36:18.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:18.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:18.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:18.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:18.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:18.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:18.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.134+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f981a2910 con 0x7f3f98108900 2024-04-25T16:36:18.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.135+0000 7f3f8e7fc640 1 -- 172.21.15.112:0/627737536 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f3f940a6a90 con 0x7f3f98108900 2024-04-25T16:36:18.134 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:18.134 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T16:36:18.134 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3f840afaf0 msgr2=0x7f3f840b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3f840afaf0 0x7f3f840b2000 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3f90000f80 tx=0x7f3f90004a40 comp rx=0 tx=0).stop 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 msgr2=0x7f3f981a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 0x7f3f981a16f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400d750 tx=0x7f3f9400dc10 comp rx=0 tx=0).stop 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 shutdown_connections 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3f840afaf0 0x7f3f840b2000 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f98103c00 0x7f3f9819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f98108900 0x7f3f981a16f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 --2- 172.21.15.112:0/627737536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f981029c0 0x7f3f9819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 >> 172.21.15.112:0/627737536 conn(0x7f3f980fdbd0 msgr2=0x7f3f98109eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 shutdown_connections 2024-04-25T16:36:18.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:18.137+0000 7f3f9f277640 1 -- 172.21.15.112:0/627737536 wait complete. 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T16:36:18.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:18.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:18 smithi116 ceph-mon[32712]: Deploying daemon mds.b on smithi116 2024-04-25T16:36:18.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:18 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.112:6840/4208559487,v1:172.21.15.112:6841/4208559487] up:boot 2024-04-25T16:36:18.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:18 smithi116 ceph-mon[32712]: fsmap 11 up:standby 2024-04-25T16:36:18.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:18 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T16:36:18.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:18 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/627737536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:19.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:18 smithi112 ceph-mon[29122]: Deploying daemon mds.b on smithi116 2024-04-25T16:36:19.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:18 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.112:6840/4208559487,v1:172.21.15.112:6841/4208559487] up:boot 2024-04-25T16:36:19.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:18 smithi112 ceph-mon[29122]: fsmap 11 up:standby 2024-04-25T16:36:19.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:18 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T16:36:19.096 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:18 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/627737536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:19.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:18 smithi133 ceph-mon[32691]: Deploying daemon mds.b on smithi116 2024-04-25T16:36:19.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:18 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.112:6840/4208559487,v1:172.21.15.112:6841/4208559487] up:boot 2024-04-25T16:36:19.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:18 smithi133 ceph-mon[32691]: fsmap 11 up:standby 2024-04-25T16:36:19.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:18 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T16:36:19.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:18 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/627737536' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:19.963 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:19 smithi116 ceph-mon[32712]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:19.963 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:19 smithi116 systemd[1]: Starting Ceph mds.b for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:36:20.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:19 smithi112 ceph-mon[29122]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:20.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:20.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:19 smithi133 ceph-mon[32691]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:20.238 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:19 smithi116 podman[63368]: 2024-04-25 16:36:19.867425373 +0000 UTC m=+0.017634361 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:36:20.238 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:19 smithi116 podman[63368]: 2024-04-25 16:36:19.97705831 +0000 UTC m=+0.127267273 container create 755e27b715b3682a4a508b8e4ee17d93f3ef39cb6144182f75f55ca8a67a706e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T16:36:20.239 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:20 smithi116 podman[63368]: 2024-04-25 16:36:20.204913818 +0000 UTC m=+0.355122789 container init 755e27b715b3682a4a508b8e4ee17d93f3ef39cb6144182f75f55ca8a67a706e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T16:36:20.239 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:20 smithi116 podman[63368]: 2024-04-25 16:36:20.207458857 +0000 UTC m=+0.357667824 container start 755e27b715b3682a4a508b8e4ee17d93f3ef39cb6144182f75f55ca8a67a706e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T16:36:20.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 -- 172.21.15.112:0/2205431690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc104690 msgr2=0x7f37bc10ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 --2- 172.21.15.112:0/2205431690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc104690 0x7f37bc10ade0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009ca0 tx=0x7f37b402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 -- 172.21.15.112:0/2205431690 shutdown_connections 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 --2- 172.21.15.112:0/2205431690 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc103b30 0x7f37bc104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 --2- 172.21.15.112:0/2205431690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc104690 0x7f37bc10ade0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 --2- 172.21.15.112:0/2205431690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc102fd0 0x7f37bc1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.271+0000 7f37c1669640 1 -- 172.21.15.112:0/2205431690 >> 172.21.15.112:0/2205431690 conn(0x7f37bc0fd370 msgr2=0x7f37bc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.272+0000 7f37c1669640 1 -- 172.21.15.112:0/2205431690 shutdown_connections 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.272+0000 7f37c1669640 1 -- 172.21.15.112:0/2205431690 wait complete. 2024-04-25T16:36:20.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.272+0000 7f37c1669640 1 Processor -- start 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.272+0000 7f37c1669640 1 -- start start 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 0x7f37bc199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc103b30 0x7f37bc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc19f410 0x7f37bc1a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10e740 con 0x7f37bc102fd0 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10e5c0 con 0x7f37bc103b30 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37bc10e8c0 con 0x7f37bc19f410 2024-04-25T16:36:20.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 0x7f37bc199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37ba7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc103b30 0x7f37bc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 0x7f37bc199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36250/0 (socket says 172.21.15.112:36250) 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 -- 172.21.15.112:0/976845693 learned_addr learned my addr 172.21.15.112:0/976845693 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37bb7fe640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc19f410 0x7f37bc1a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 -- 172.21.15.112:0/976845693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc19f410 msgr2=0x7f37bc1a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc19f410 0x7f37bc1a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 -- 172.21.15.112:0/976845693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc103b30 msgr2=0x7f37bc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc103b30 0x7f37bc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 -- 172.21.15.112:0/976845693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37bc1a1ea0 con 0x7f37bc102fd0 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37bb7fe640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc19f410 0x7f37bc1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37ba7fc640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc103b30 0x7f37bc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37baffd640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 0x7f37bc199f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00d770 tx=0x7f37ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:20.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac017070 con 0x7f37bc102fd0 2024-04-25T16:36:20.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37ac009c30 con 0x7f37bc102fd0 2024-04-25T16:36:20.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37bc1a2190 con 0x7f37bc102fd0 2024-04-25T16:36:20.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00f620 con 0x7f37bc102fd0 2024-04-25T16:36:20.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.273+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37bc1a26e0 con 0x7f37bc102fd0 2024-04-25T16:36:20.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.275+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37bc06b3a0 con 0x7f37bc102fd0 2024-04-25T16:36:20.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.276+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f37ac01a070 con 0x7f37bc102fd0 2024-04-25T16:36:20.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.277+0000 7f37a3fff640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f37a40afaf0 0x7f37a40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:20.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.277+0000 7f37ba7fc640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f37a40afaf0 0x7f37a40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:20.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.277+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f37ac0de2f0 con 0x7f37bc102fd0 2024-04-25T16:36:20.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.277+0000 7f37ba7fc640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f37a40afaf0 0x7f37a40b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f37bc19aff0 tx=0x7f37b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:20.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.281+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f37ac009de0 con 0x7f37bc102fd0 2024-04-25T16:36:20.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.507+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37bc1a2a00 con 0x7f37bc102fd0 2024-04-25T16:36:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.508+0000 7f37a3fff640 1 -- 172.21.15.112:0/976845693 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7f37ac0a7350 con 0x7f37bc102fd0 2024-04-25T16:36:20.507 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:20.507 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T16:36:20.507 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 12 2024-04-25T16:36:20.508 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:20 smithi116 ceph-mds[63387]: -- 172.21.15.116:0/69229481 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618b28c1a40 con 0x5618b2909600 2024-04-25T16:36:20.508 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: starting mds.b at 2024-04-25T16:36:20.508 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:20 smithi116 bash[63368]: 755e27b715b3682a4a508b8e4ee17d93f3ef39cb6144182f75f55ca8a67a706e 2024-04-25T16:36:20.508 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:20 smithi116 systemd[1]: Started Ceph mds.b for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f37a40afaf0 msgr2=0x7f37a40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:20.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f37a40afaf0 0x7f37a40b2000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f37bc19aff0 tx=0x7f37b000b410 comp rx=0 tx=0).stop 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 msgr2=0x7f37bc199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 0x7f37bc199f50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f37ac00d770 tx=0x7f37ac00dc30 comp rx=0 tx=0).stop 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 shutdown_connections 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f37a40afaf0 0x7f37a40b2000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37bc19f410 0x7f37bc1a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37bc103b30 0x7f37bc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 --2- 172.21.15.112:0/976845693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37bc102fd0 0x7f37bc199f50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 >> 172.21.15.112:0/976845693 conn(0x7f37bc0fd370 msgr2=0x7f37bc0fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 shutdown_connections 2024-04-25T16:36:20.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:20.510+0000 7f37c1669640 1 -- 172.21.15.112:0/976845693 wait complete. 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/976845693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] up:boot 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: fsmap 12 up:standby 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:36:21.632 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:21 smithi112 ceph-mon[29122]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:21.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/976845693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] up:boot 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: fsmap 12 up:standby 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:36:21.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:21 smithi133 ceph-mon[32691]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:21.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/976845693' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] up:boot 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: fsmap 12 up:standby 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:36:21.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:21 smithi116 ceph-mon[32712]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:22.522 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:22.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/2223459613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14108900 msgr2=0x7ffa1410ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:22.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/2223459613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14108900 0x7ffa1410ad20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009ca0 tx=0x7ffa0c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/2223459613 shutdown_connections 2024-04-25T16:36:22.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/2223459613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa14103c00 0x7ffa14104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/2223459613 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14108900 0x7ffa1410ad20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/2223459613 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa141029c0 0x7ffa141036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/2223459613 >> 172.21.15.112:0/2223459613 conn(0x7ffa140fdbd0 msgr2=0x7ffa14100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/2223459613 shutdown_connections 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.634+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/2223459613 wait complete. 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 Processor -- start 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 -- start start 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa141029c0 0x7ffa1419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 0x7ffa1419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa14108900 0x7ffa141a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1410e4f0 con 0x7ffa14103c00 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1410e370 con 0x7ffa141029c0 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1b0f0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1410e670 con 0x7ffa14108900 2024-04-25T16:36:22.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa13fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 0x7ffa1419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa13fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 0x7ffa1419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36270/0 (socket says 172.21.15.112:36270) 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa13fff640 1 -- 172.21.15.112:0/1578136250 learned_addr learned my addr 172.21.15.112:0/1578136250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa1968d640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa14108900 0x7ffa141a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.635+0000 7ffa18e8c640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa141029c0 0x7ffa1419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa13fff640 1 -- 172.21.15.112:0/1578136250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa14108900 msgr2=0x7ffa141a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa13fff640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa14108900 0x7ffa141a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa13fff640 1 -- 172.21.15.112:0/1578136250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa141029c0 msgr2=0x7ffa1419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa13fff640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa141029c0 0x7ffa1419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa13fff640 1 -- 172.21.15.112:0/1578136250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa141a1cf0 con 0x7ffa14103c00 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa13fff640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 0x7ffa1419a6f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000e9f0 tx=0x7ffa0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0000cd80 con 0x7ffa14103c00 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa141a1fe0 con 0x7ffa14103c00 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa000054d0 con 0x7ffa14103c00 2024-04-25T16:36:22.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00010690 con 0x7ffa14103c00 2024-04-25T16:36:22.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.636+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141a2450 con 0x7ffa14103c00 2024-04-25T16:36:22.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.637+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa141a2710 con 0x7ffa14103c00 2024-04-25T16:36:22.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.639+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7ffa00004e30 con 0x7ffa14103c00 2024-04-25T16:36:22.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.639+0000 7ffa11ffb640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ffa040afaf0 0x7ffa040b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:22.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.639+0000 7ffa18e8c640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ffa040afaf0 0x7ffa040b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:22.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.639+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ffa000ded90 con 0x7ffa14103c00 2024-04-25T16:36:22.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.640+0000 7ffa18e8c640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ffa040afaf0 0x7ffa040b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa141035c0 tx=0x7ffa08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:22.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.648+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa00053ff0 con 0x7ffa14103c00 2024-04-25T16:36:22.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.873+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa141a28e0 con 0x7ffa14103c00 2024-04-25T16:36:22.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.874+0000 7ffa11ffb640 1 -- 172.21.15.112:0/1578136250 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7ffa000a7e10 con 0x7ffa14103c00 2024-04-25T16:36:22.873 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:22.873 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24472,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24484,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24524,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T16:36:22.873 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 13 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ffa040afaf0 msgr2=0x7ffa040b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ffa040afaf0 0x7ffa040b2000 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa141035c0 tx=0x7ffa08005f90 comp rx=0 tx=0).stop 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 msgr2=0x7ffa1419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 0x7ffa1419a6f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa0000e9f0 tx=0x7ffa0000edf0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 shutdown_connections 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ffa040afaf0 0x7ffa040b2000 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa14108900 0x7ffa141a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffa14103c00 0x7ffa1419a6f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 --2- 172.21.15.112:0/1578136250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa141029c0 0x7ffa1419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:22.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.876+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 >> 172.21.15.112:0/1578136250 conn(0x7ffa140fdbd0 msgr2=0x7ffa14109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:22.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.877+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 shutdown_connections 2024-04-25T16:36:22.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:22.877+0000 7ffa1b0f0640 1 -- 172.21.15.112:0/1578136250 wait complete. 2024-04-25T16:36:22.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 -- 172.21.15.112:0/2319048157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 msgr2=0x7fbad810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 --2- 172.21.15.112:0/2319048157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad810ad20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009ca0 tx=0x7fbad002cac0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 -- 172.21.15.112:0/2319048157 shutdown_connections 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 --2- 172.21.15.112:0/2319048157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 0x7fbad8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 --2- 172.21.15.112:0/2319048157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad810ad20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 --2- 172.21.15.112:0/2319048157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad81029c0 0x7fbad81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 -- 172.21.15.112:0/2319048157 >> 172.21.15.112:0/2319048157 conn(0x7fbad80fdbd0 msgr2=0x7fbad8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 -- 172.21.15.112:0/2319048157 shutdown_connections 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.004+0000 7fbadf196640 1 -- 172.21.15.112:0/2319048157 wait complete. 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 Processor -- start 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 -- start start 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad81029c0 0x7fbad819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 0x7fbad819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad81a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810d3e0 con 0x7fbad8108900 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810d260 con 0x7fbad81029c0 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadf196640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbad810d560 con 0x7fbad8103c00 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadd733640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad81a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadd733640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad81a3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36280/0 (socket says 172.21.15.112:36280) 2024-04-25T16:36:23.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadd733640 1 -- 172.21.15.112:0/1264797084 learned_addr learned my addr 172.21.15.112:0/1264797084 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.005+0000 7fbadcf32640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad81029c0 0x7fbad819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbad7fff640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 0x7fbad819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadd733640 1 -- 172.21.15.112:0/1264797084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 msgr2=0x7fbad819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadd733640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 0x7fbad819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadd733640 1 -- 172.21.15.112:0/1264797084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad81029c0 msgr2=0x7fbad819c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadd733640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad81029c0 0x7fbad819c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadd733640 1 -- 172.21.15.112:0/1264797084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad81a3ed0 con 0x7fbad8108900 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbad7fff640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 0x7fbad819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadd733640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad81a3860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbad0038f40 tx=0x7fbad0038bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0045070 con 0x7fbad8108900 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad81a4160 con 0x7fbad8108900 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbad002e620 con 0x7fbad8108900 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad0040690 con 0x7fbad8108900 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.006+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad81a46a0 con 0x7fbad8108900 2024-04-25T16:36:23.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.007+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad8107710 con 0x7fbad8108900 2024-04-25T16:36:23.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.009+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fbad002ea90 con 0x7fbad8108900 2024-04-25T16:36:23.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.009+0000 7fbad5ffb640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbac80afaf0 0x7fbac80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.010+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fbad00ff380 con 0x7fbad8108900 2024-04-25T16:36:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.010+0000 7fbadcf32640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbac80afaf0 0x7fbac80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.010+0000 7fbadcf32640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbac80afaf0 0x7fbac80b2000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbad81035c0 tx=0x7fbacc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.017+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbad0034030 con 0x7fbad8108900 2024-04-25T16:36:23.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.195+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbad802dd20 con 0x7fbad8108900 2024-04-25T16:36:23.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.196+0000 7fbad5ffb640 1 -- 172.21.15.112:0/1264797084 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18536 (secure 0 0 0) 0x7fbad00c8400 con 0x7fbad8108900 2024-04-25T16:36:23.195 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:23.195 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":71,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:35:36.145733+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:36:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbac80afaf0 msgr2=0x7fbac80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbac80afaf0 0x7fbac80b2000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbad81035c0 tx=0x7fbacc00b390 comp rx=0 tx=0).stop 2024-04-25T16:36:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 msgr2=0x7fbad81a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad81a3860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbad0038f40 tx=0x7fbad0038bd0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 shutdown_connections 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbac80afaf0 0x7fbac80b2000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad8103c00 0x7fbad819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbad8108900 0x7fbad81a3860 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 --2- 172.21.15.112:0/1264797084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad81029c0 0x7fbad819c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 >> 172.21.15.112:0/1264797084 conn(0x7fbad80fdbd0 msgr2=0x7fbad81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 shutdown_connections 2024-04-25T16:36:23.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.198+0000 7fbadf196640 1 -- 172.21.15.112:0/1264797084 wait complete. 2024-04-25T16:36:23.208 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:36:23.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:36:23.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 -- 172.21.15.112:0/1068412493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b38102170 msgr2=0x7f4b38102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/1068412493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b38102170 0x7f4b38102e70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4b28009900 tx=0x7f4b2802a410 comp rx=0 tx=0).stop 2024-04-25T16:36:23.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 -- 172.21.15.112:0/1068412493 shutdown_connections 2024-04-25T16:36:23.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/1068412493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b381033b0 0x7f4b381037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/1068412493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b381080b0 0x7f4b3810a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/1068412493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b38102170 0x7f4b38102e70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 -- 172.21.15.112:0/1068412493 >> 172.21.15.112:0/1068412493 conn(0x7f4b380fd340 msgr2=0x7f4b380ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 -- 172.21.15.112:0/1068412493 shutdown_connections 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.327+0000 7f4b3ffce640 1 -- 172.21.15.112:0/1068412493 wait complete. 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 Processor -- start 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 -- start start 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b38102170 0x7f4b3810ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b381033b0 0x7f4b3819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 0x7f4b381a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3810e510 con 0x7f4b381080b0 2024-04-25T16:36:23.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3810e390 con 0x7f4b381033b0 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3ffce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3810e690 con 0x7f4b38102170 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3e56b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 0x7f4b381a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3e56b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 0x7f4b381a1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36292/0 (socket says 172.21.15.112:36292) 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.328+0000 7f4b3e56b640 1 -- 172.21.15.112:0/3075173332 learned_addr learned my addr 172.21.15.112:0/3075173332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3dd6a640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b38102170 0x7f4b3810ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3d569640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b381033b0 0x7f4b3819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3e56b640 1 -- 172.21.15.112:0/3075173332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b38102170 msgr2=0x7f4b3810ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3e56b640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b38102170 0x7f4b3810ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3e56b640 1 -- 172.21.15.112:0/3075173332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b381033b0 msgr2=0x7f4b3819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3e56b640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b381033b0 0x7f4b3819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3e56b640 1 -- 172.21.15.112:0/3075173332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b381a1d70 con 0x7f4b381080b0 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3e56b640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 0x7f4b381a1550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000eb10 tx=0x7f4b3000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30004b20 con 0x7f4b381080b0 2024-04-25T16:36:23.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b30004cc0 con 0x7f4b381080b0 2024-04-25T16:36:23.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b30010670 con 0x7f4b381080b0 2024-04-25T16:36:23.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b381a2060 con 0x7f4b381080b0 2024-04-25T16:36:23.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.329+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b381a2560 con 0x7f4b381080b0 2024-04-25T16:36:23.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.331+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b38106ec0 con 0x7f4b381080b0 2024-04-25T16:36:23.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.331+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f4b30005020 con 0x7f4b381080b0 2024-04-25T16:36:23.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.332+0000 7f4b2effd640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4b240afaf0 0x7f4b240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.332+0000 7f4b3dd6a640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4b240afaf0 0x7f4b240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.332+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f4b300ded70 con 0x7f4b381080b0 2024-04-25T16:36:23.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.333+0000 7f4b3dd6a640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4b240afaf0 0x7f4b240b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005ab0 tx=0x7f4b28031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.337+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b300a7df0 con 0x7f4b381080b0 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:23.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:36:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1578136250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:23 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1264797084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1578136250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:23 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1264797084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.514 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1578136250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.515 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:23 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1264797084' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.526+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b380637a0 con 0x7f4b381080b0 2024-04-25T16:36:23.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.526+0000 7f4b2effd640 1 -- 172.21.15.112:0/3075173332 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f4b300aee20 con 0x7f4b381080b0 2024-04-25T16:36:23.525 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4b240afaf0 msgr2=0x7f4b240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4b240afaf0 0x7f4b240b2000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b28005ab0 tx=0x7f4b28031400 comp rx=0 tx=0).stop 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 msgr2=0x7f4b381a1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 0x7f4b381a1550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000eb10 tx=0x7f4b3000c2d0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 shutdown_connections 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4b381080b0 0x7f4b381a1550 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b381033b0 0x7f4b3819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4b240afaf0 0x7f4b240b2000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 --2- 172.21.15.112:0/3075173332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b38102170 0x7f4b3810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 >> 172.21.15.112:0/3075173332 conn(0x7f4b380fd340 msgr2=0x7f4b38104090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 shutdown_connections 2024-04-25T16:36:23.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.528+0000 7f4b3ffce640 1 -- 172.21.15.112:0/3075173332 wait complete. 2024-04-25T16:36:23.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 -- 172.21.15.112:0/2688246991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704103c40 msgr2=0x7ff704104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 --2- 172.21.15.112:0/2688246991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704103c40 0x7ff704104050 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009860 tx=0x7ff6f402a340 comp rx=0 tx=0).stop 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 -- 172.21.15.112:0/2688246991 shutdown_connections 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 --2- 172.21.15.112:0/2688246991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff704108940 0x7ff70410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 --2- 172.21.15.112:0/2688246991 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704103c40 0x7ff704104050 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 --2- 172.21.15.112:0/2688246991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff704102a00 0x7ff704103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.647+0000 7ff70be69640 1 -- 172.21.15.112:0/2688246991 >> 172.21.15.112:0/2688246991 conn(0x7ff7040fdbf0 msgr2=0x7ff7041000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 -- 172.21.15.112:0/2688246991 shutdown_connections 2024-04-25T16:36:23.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 -- 172.21.15.112:0/2688246991 wait complete. 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 Processor -- start 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 -- start start 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff704102a00 0x7ff7040758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff704103c40 0x7ff70419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 0x7ff7041a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff70410e500 con 0x7ff704108940 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff70410e380 con 0x7ff704102a00 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.648+0000 7ff70be69640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff70410e680 con 0x7ff704103c40 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 0x7ff7041a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 0x7ff7041a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36314/0 (socket says 172.21.15.112:36314) 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff709c05640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff704102a00 0x7ff7040758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 -- 172.21.15.112:0/2616344552 learned_addr learned my addr 172.21.15.112:0/2616344552 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff709404640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff704103c40 0x7ff70419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 -- 172.21.15.112:0/2616344552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff704103c40 msgr2=0x7ff70419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff704103c40 0x7ff70419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 -- 172.21.15.112:0/2616344552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff704102a00 msgr2=0x7ff7040758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff704102a00 0x7ff7040758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 -- 172.21.15.112:0/2616344552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7041a1cf0 con 0x7ff704108940 2024-04-25T16:36:23.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70a406640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 0x7ff7041a14d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00efc0 tx=0x7ff6fc00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc003970 con 0x7ff704108940 2024-04-25T16:36:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6fc004020 con 0x7ff704108940 2024-04-25T16:36:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7041a1fe0 con 0x7ff704108940 2024-04-25T16:36:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc00b940 con 0x7ff704108940 2024-04-25T16:36:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.649+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041a23f0 con 0x7ff704108940 2024-04-25T16:36:23.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.650+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff70406a1c0 con 0x7ff704108940 2024-04-25T16:36:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.652+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7ff6fc01d020 con 0x7ff704108940 2024-04-25T16:36:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.652+0000 7ff6faffd640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6f00afaf0 0x7ff6f00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.652+0000 7ff709c05640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6f00afaf0 0x7ff6f00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.652+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff6fc0dec20 con 0x7ff704108940 2024-04-25T16:36:23.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.653+0000 7ff709c05640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6f00afaf0 0x7ff6f00b2000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff704103600 tx=0x7ff700005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.656+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff70406a1c0 con 0x7ff704108940 2024-04-25T16:36:23.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.850+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff70402dcb0 con 0x7ff704108940 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.850+0000 7ff6faffd640 1 -- 172.21.15.112:0/2616344552 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7ff70402dcb0 con 0x7ff704108940 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-25T16:36:23.849 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:36:23.850 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-25T16:36:23.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6f00afaf0 msgr2=0x7ff6f00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6f00afaf0 0x7ff6f00b2000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff704103600 tx=0x7ff700005f90 comp rx=0 tx=0).stop 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 msgr2=0x7ff7041a14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 0x7ff7041a14d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00efc0 tx=0x7ff6fc00c800 comp rx=0 tx=0).stop 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 shutdown_connections 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6f00afaf0 0x7ff6f00b2000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff704103c40 0x7ff70419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff704108940 0x7ff7041a14d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 --2- 172.21.15.112:0/2616344552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff704102a00 0x7ff7040758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 >> 172.21.15.112:0/2616344552 conn(0x7ff7040fdbf0 msgr2=0x7ff704104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 shutdown_connections 2024-04-25T16:36:23.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.853+0000 7ff70be69640 1 -- 172.21.15.112:0/2616344552 wait complete. 2024-04-25T16:36:23.864 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T16:36:23.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T16:36:23.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 -- 172.21.15.112:0/4157850812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c1045d0 msgr2=0x7f271c1069f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 --2- 172.21.15.112:0/4157850812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c1045d0 0x7f271c1069f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2710009900 tx=0x7f271002a410 comp rx=0 tx=0).stop 2024-04-25T16:36:23.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 -- 172.21.15.112:0/4157850812 shutdown_connections 2024-04-25T16:36:23.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 --2- 172.21.15.112:0/4157850812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271c106f30 0x7f271c109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 --2- 172.21.15.112:0/4157850812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c109890 0x7f271c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 --2- 172.21.15.112:0/4157850812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c1045d0 0x7f271c1069f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 -- 172.21.15.112:0/4157850812 >> 172.21.15.112:0/4157850812 conn(0x7f271c0fdbd0 msgr2=0x7f271c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 -- 172.21.15.112:0/4157850812 shutdown_connections 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.990+0000 7f2723184640 1 -- 172.21.15.112:0/4157850812 wait complete. 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 Processor -- start 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 -- start start 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c1045d0 0x7f271c103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271c106f30 0x7f271c1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109890 0x7f271c102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10f5e0 con 0x7f271c109890 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10f460 con 0x7f271c1045d0 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2723184640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f271c10f760 con 0x7f271c106f30 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2721721640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109890 0x7f271c102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2720f20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c1045d0 0x7f271c103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2721721640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109890 0x7f271c102600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36336/0 (socket says 172.21.15.112:36336) 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f2721721640 1 -- 172.21.15.112:0/752597192 learned_addr learned my addr 172.21.15.112:0/752597192 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.991+0000 7f271bfff640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271c106f30 0x7f271c1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2720f20640 1 -- 172.21.15.112:0/752597192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271c106f30 msgr2=0x7f271c1020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2720f20640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271c106f30 0x7f271c1020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2720f20640 1 -- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109890 msgr2=0x7f271c102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2720f20640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109890 0x7f271c102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2720f20640 1 -- 172.21.15.112:0/752597192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c102d30 con 0x7f271c1045d0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2720f20640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c1045d0 0x7f271c103a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2710006010 tx=0x7f2710004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271003f070 con 0x7f271c1045d0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c1a4650 con 0x7f271c1045d0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2710031910 con 0x7f271c1045d0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710035070 con 0x7f271c1045d0 2024-04-25T16:36:23.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.992+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a4ab0 con 0x7f271c1045d0 2024-04-25T16:36:23.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.993+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c06c2a0 con 0x7f271c1045d0 2024-04-25T16:36:23.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.995+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f271003bbb0 con 0x7f271c1045d0 2024-04-25T16:36:23.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.996+0000 7f2719ffb640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f270c0afaf0 0x7f270c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:23.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.996+0000 7f271bfff640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f270c0afaf0 0x7f270c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:23.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.996+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f27100ff010 con 0x7f271c1045d0 2024-04-25T16:36:23.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.997+0000 7f271bfff640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f270c0afaf0 0x7f270c0b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f271c1033c0 tx=0x7f2708005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:23.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:23.999+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27100c7a40 con 0x7f271c1045d0 2024-04-25T16:36:24.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:24.199+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f271c1a4ce0 con 0x7f271c1045d0 2024-04-25T16:36:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3075173332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:36:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2616344552' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/752597192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T16:36:24.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:24 smithi116 ceph-mon[32712]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3075173332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2616344552' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:36:24.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/752597192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T16:36:24.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:24 smithi112 ceph-mon[29122]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3075173332' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2616344552' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.616 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.617 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.617 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:36:24.617 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:36:24.617 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:24.617 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/752597192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T16:36:24.617 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:24 smithi133 ceph-mon[32691]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T16:36:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.198+0000 7f2719ffb640 1 -- 172.21.15.112:0/752597192 <== mon.2 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f27100cbd40 con 0x7f271c1045d0 2024-04-25T16:36:25.197 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f270c0afaf0 msgr2=0x7f270c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f270c0afaf0 0x7f270c0b2000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f271c1033c0 tx=0x7f2708005f90 comp rx=0 tx=0).stop 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c1045d0 msgr2=0x7f271c103a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c1045d0 0x7f271c103a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2710006010 tx=0x7f2710004270 comp rx=0 tx=0).stop 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 shutdown_connections 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f270c0afaf0 0x7f270c0b2000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271c106f30 0x7f271c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f271c109890 0x7f271c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 --2- 172.21.15.112:0/752597192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f271c1045d0 0x7f271c103a20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 >> 172.21.15.112:0/752597192 conn(0x7f271c0fdbd0 msgr2=0x7f271c104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 shutdown_connections 2024-04-25T16:36:25.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.201+0000 7f2723184640 1 -- 172.21.15.112:0/752597192 wait complete. 2024-04-25T16:36:25.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T16:36:25.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 -- 172.21.15.112:0/2992273389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2741029c0 msgr2=0x7ff2741036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:25.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 --2- 172.21.15.112:0/2992273389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2741029c0 0x7ff2741036c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff264009900 tx=0x7ff26402a410 comp rx=0 tx=0).stop 2024-04-25T16:36:25.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 -- 172.21.15.112:0/2992273389 shutdown_connections 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 --2- 172.21.15.112:0/2992273389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff274103c00 0x7ff274104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 --2- 172.21.15.112:0/2992273389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff274108900 0x7ff27410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 --2- 172.21.15.112:0/2992273389 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2741029c0 0x7ff2741036c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 -- 172.21.15.112:0/2992273389 >> 172.21.15.112:0/2992273389 conn(0x7ff2740fdbd0 msgr2=0x7ff274100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 -- 172.21.15.112:0/2992273389 shutdown_connections 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.335+0000 7ff2799b4640 1 -- 172.21.15.112:0/2992273389 wait complete. 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 Processor -- start 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 -- start start 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741029c0 0x7ff27419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff274103c00 0x7ff27419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 0x7ff2741a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff27410e4c0 con 0x7ff274108900 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff27410e340 con 0x7ff274103c00 2024-04-25T16:36:25.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2799b4640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff27410e640 con 0x7ff2741029c0 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 0x7ff2741a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 0x7ff2741a16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36346/0 (socket says 172.21.15.112:36346) 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 -- 172.21.15.112:0/3510954891 learned_addr learned my addr 172.21.15.112:0/3510954891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff272ffd640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741029c0 0x7ff27419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff26bfff640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff274103c00 0x7ff27419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 -- 172.21.15.112:0/3510954891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741029c0 msgr2=0x7ff27419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741029c0 0x7ff27419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 -- 172.21.15.112:0/3510954891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff274103c00 msgr2=0x7ff27419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff274103c00 0x7ff27419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 -- 172.21.15.112:0/3510954891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2741a1d60 con 0x7ff274108900 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.336+0000 7ff2737fe640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 0x7ff2741a16f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00eaa0 tx=0x7ff26c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.337+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00cdb0 con 0x7ff274108900 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.337+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff26c005020 con 0x7ff274108900 2024-04-25T16:36:25.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.337+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2741a1ff0 con 0x7ff274108900 2024-04-25T16:36:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.337+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0116d0 con 0x7ff274108900 2024-04-25T16:36:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.337+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2741a2460 con 0x7ff274108900 2024-04-25T16:36:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.338+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2741a26f0 con 0x7ff274108900 2024-04-25T16:36:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.339+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7ff26c0051d0 con 0x7ff274108900 2024-04-25T16:36:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.339+0000 7ff2789b2640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff2600afaf0 0x7ff2600b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.340+0000 7ff272ffd640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff2600afaf0 0x7ff2600b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.340+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7ff26c0dee30 con 0x7ff274108900 2024-04-25T16:36:25.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.340+0000 7ff272ffd640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff2600afaf0 0x7ff2600b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff264005ab0 tx=0x7ff264031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:25.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.344+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff26c016030 con 0x7ff274108900 2024-04-25T16:36:25.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:25.520+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff2741a2e00 con 0x7ff274108900 2024-04-25T16:36:26.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.191+0000 7ff2789b2640 1 -- 172.21.15.112:0/3510954891 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7ff26c0a7d40 con 0x7ff274108900 2024-04-25T16:36:26.190 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-04-25T16:36:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.194+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff2600afaf0 msgr2=0x7ff2600b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.194+0000 7ff2799b4640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff2600afaf0 0x7ff2600b2000 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff264005ab0 tx=0x7ff264031400 comp rx=0 tx=0).stop 2024-04-25T16:36:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.194+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 msgr2=0x7ff2741a16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:26.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.194+0000 7ff2799b4640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 0x7ff2741a16f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00eaa0 tx=0x7ff26c00ef60 comp rx=0 tx=0).stop 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 shutdown_connections 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff274108900 0x7ff2741a16f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff274103c00 0x7ff27419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff2600afaf0 0x7ff2600b2000 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 --2- 172.21.15.112:0/3510954891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741029c0 0x7ff27419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 >> 172.21.15.112:0/3510954891 conn(0x7ff2740fdbd0 msgr2=0x7ff274109ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 shutdown_connections 2024-04-25T16:36:26.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.195+0000 7ff2799b4640 1 -- 172.21.15.112:0/3510954891 wait complete. 2024-04-25T16:36:26.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T16:36:26.326 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:26.326 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:26 smithi116 ceph-mon[32712]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:26.326 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:26 smithi116 ceph-mon[32712]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T16:36:26.326 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:26 smithi116 ceph-mon[32712]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T16:36:26.326 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:26 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3510954891' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 -- 172.21.15.112:0/362082124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e801029e0 msgr2=0x7f4e801036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/362082124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e801029e0 0x7f4e801036e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009900 tx=0x7f4e7002a410 comp rx=0 tx=0).stop 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 -- 172.21.15.112:0/362082124 shutdown_connections 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/362082124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e80103c20 0x7f4e80104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/362082124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e80108920 0x7f4e8010ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/362082124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e801029e0 0x7f4e801036e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 -- 172.21.15.112:0/362082124 >> 172.21.15.112:0/362082124 conn(0x7f4e800fdbd0 msgr2=0x7f4e80100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 -- 172.21.15.112:0/362082124 shutdown_connections 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 -- 172.21.15.112:0/362082124 wait complete. 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 Processor -- start 2024-04-25T16:36:26.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.335+0000 7f4e87f6c640 1 -- start start 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e801029e0 0x7f4e8006a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 0x7f4e8019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e80108920 0x7f4e801a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010e530 con 0x7f4e80103c20 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010e3b0 con 0x7f4e801029e0 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8010e6b0 con 0x7f4e80108920 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 0x7f4e8019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 0x7f4e8019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52354/0 (socket says 172.21.15.112:52354) 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 -- 172.21.15.112:0/403845828 learned_addr learned my addr 172.21.15.112:0/403845828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85d08640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e801029e0 0x7f4e8006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e86509640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e80108920 0x7f4e801a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 -- 172.21.15.112:0/403845828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e80108920 msgr2=0x7f4e801a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:26.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e80108920 0x7f4e801a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 -- 172.21.15.112:0/403845828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e801029e0 msgr2=0x7f4e8006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e801029e0 0x7f4e8006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 -- 172.21.15.112:0/403845828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e801a20a0 con 0x7f4e80103c20 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e85507640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 0x7f4e8019f330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e9f0 tx=0x7f4e7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00cd80 con 0x7f4e80103c20 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e7c0054d0 con 0x7f4e80103c20 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c010690 con 0x7f4e80103c20 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e801a2390 con 0x7f4e80103c20 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.336+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e801a28e0 con 0x7f4e80103c20 2024-04-25T16:36:26.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.337+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e80107730 con 0x7f4e80103c20 2024-04-25T16:36:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.339+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f4e7c004e30 con 0x7f4e80103c20 2024-04-25T16:36:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.340+0000 7f4e76ffd640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4e6c0afb70 0x7f4e6c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.340+0000 7f4e85d08640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4e6c0afb70 0x7f4e6c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.340+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7879+0+0 (secure 0 0 0) 0x7f4e7c0df0a0 con 0x7f4e80103c20 2024-04-25T16:36:26.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.340+0000 7f4e85d08640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4e6c0afb70 0x7f4e6c0b2080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002aa00 tx=0x7f4e700314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:26.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.344+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e7c053ff0 con 0x7f4e80103c20 2024-04-25T16:36:26.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:26.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:26 smithi112 ceph-mon[29122]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:26.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:26 smithi112 ceph-mon[29122]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T16:36:26.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:26 smithi112 ceph-mon[29122]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T16:36:26.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:26 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3510954891' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T16:36:26.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:36:26.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:26 smithi133 ceph-mon[32691]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:26.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:26 smithi133 ceph-mon[32691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T16:36:26.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:26 smithi133 ceph-mon[32691]: osdmap e72: 12 total, 12 up, 12 in 2024-04-25T16:36:26.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:26 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3510954891' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T16:36:26.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:26.561+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4e8002dd00 con 0x7f4e80103c20 2024-04-25T16:36:27.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.192+0000 7f4e76ffd640 1 -- 172.21.15.112:0/403845828 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f4e7c0a7e30 con 0x7f4e80103c20 2024-04-25T16:36:27.191 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T16:36:27.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4e6c0afb70 msgr2=0x7f4e6c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4e6c0afb70 0x7f4e6c0b2080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002aa00 tx=0x7f4e700314a0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 msgr2=0x7f4e8019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 0x7f4e8019f330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e9f0 tx=0x7f4e7c00edf0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 shutdown_connections 2024-04-25T16:36:27.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4e6c0afb70 0x7f4e6c0b2080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e80108920 0x7f4e801a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4e80103c20 0x7f4e8019f330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 --2- 172.21.15.112:0/403845828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e801029e0 0x7f4e8006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.194+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 >> 172.21.15.112:0/403845828 conn(0x7f4e800fdbd0 msgr2=0x7f4e80104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:27.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.195+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 shutdown_connections 2024-04-25T16:36:27.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.195+0000 7f4e87f6c640 1 -- 172.21.15.112:0/403845828 wait complete. 2024-04-25T16:36:27.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:36:27.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 -- 172.21.15.112:0/2883461197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f00102180 msgr2=0x7f3f00102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/2883461197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f00102180 0x7f3f00102e80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009900 tx=0x7f3ef002a410 comp rx=0 tx=0).stop 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 -- 172.21.15.112:0/2883461197 shutdown_connections 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/2883461197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f001080c0 0x7f3f0010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/2883461197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f001037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/2883461197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f00102180 0x7f3f00102e80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 -- 172.21.15.112:0/2883461197 >> 172.21.15.112:0/2883461197 conn(0x7f3f000fd3b0 msgr2=0x7f3f000ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 -- 172.21.15.112:0/2883461197 shutdown_connections 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.318+0000 7f3f05bd6640 1 -- 172.21.15.112:0/2883461197 wait complete. 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 Processor -- start 2024-04-25T16:36:27.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 -- start start 2024-04-25T16:36:27.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00102180 0x7f3f0019a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f0019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f001080c0 0x7f3f001a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010d3d0 con 0x7f3f001080c0 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010d250 con 0x7f3f001033c0 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3f05bd6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0010d550 con 0x7f3f00102180 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.319+0000 7f3efffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f001080c0 0x7f3f001a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f0019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f001080c0 0x7f3f001a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52374/0 (socket says 172.21.15.112:52374) 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f0019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:52354/0 (socket says 172.21.15.112:52354) 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efffff640 1 -- 172.21.15.112:0/3248229939 learned_addr learned my addr 172.21.15.112:0/3248229939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3eff7fe640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00102180 0x7f3f0019a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 -- 172.21.15.112:0/3248229939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00102180 msgr2=0x7f3f0019a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00102180 0x7f3f0019a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 -- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f001080c0 msgr2=0x7f3f001a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f001080c0 0x7f3f001a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 -- 172.21.15.112:0/3248229939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001a1cd0 con 0x7f3f001033c0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3eff7fe640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00102180 0x7f3f0019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efeffd640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f0019a780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400ea10 tx=0x7f3ef400eed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400cce0 con 0x7f3f001033c0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef4005020 con 0x7f3f001033c0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f001a1fc0 con 0x7f3f001033c0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4004a90 con 0x7f3f001033c0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.320+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f001a2420 con 0x7f3f001033c0 2024-04-25T16:36:27.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.322+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f00106ed0 con 0x7f3f001033c0 2024-04-25T16:36:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.323+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f3ef40051d0 con 0x7f3f001033c0 2024-04-25T16:36:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.323+0000 7f3efcff9640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ee80afaf0 0x7f3ee80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.323+0000 7f3eff7fe640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ee80afaf0 0x7f3ee80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.323+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f3ef40de810 con 0x7f3f001033c0 2024-04-25T16:36:27.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.324+0000 7f3eff7fe640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ee80afaf0 0x7f3ee80b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009230 tx=0x7f3ef0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:27.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.328+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ef40a74e0 con 0x7f3f001033c0 2024-04-25T16:36:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:27 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3510954891' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T16:36:27.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:27 smithi133 ceph-mon[32691]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T16:36:27.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:27 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/403845828' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T16:36:27.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:27 smithi133 ceph-mon[32691]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T16:36:27.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:27 smithi133 ceph-mon[32691]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T16:36:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:27 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3510954891' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T16:36:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:27 smithi116 ceph-mon[32712]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T16:36:27.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:27 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/403845828' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T16:36:27.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:27 smithi116 ceph-mon[32712]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T16:36:27.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:27 smithi116 ceph-mon[32712]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T16:36:27.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.520+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f000637a0 con 0x7f3f001033c0 2024-04-25T16:36:27.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.522+0000 7f3efcff9640 1 -- 172.21.15.112:0/3248229939 <== mon.2 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22312 (secure 0 0 0) 0x7f3ef40a6de0 con 0x7f3f001033c0 2024-04-25T16:36:27.521 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:27.521 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:36:26.563240+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ee80afaf0 msgr2=0x7f3ee80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ee80afaf0 0x7f3ee80b2000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009230 tx=0x7f3ef0000bc0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 msgr2=0x7f3f0019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f0019a780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400ea10 tx=0x7f3ef400eed0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 shutdown_connections 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f001080c0 0x7f3f001a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f001033c0 0x7f3f0019a780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ee80afaf0 0x7f3ee80b2000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 --2- 172.21.15.112:0/3248229939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f00102180 0x7f3f0019a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 >> 172.21.15.112:0/3248229939 conn(0x7f3f000fd3b0 msgr2=0x7f3f001040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 shutdown_connections 2024-04-25T16:36:27.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.524+0000 7f3f05bd6640 1 -- 172.21.15.112:0/3248229939 wait complete. 2024-04-25T16:36:27.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:27 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3510954891' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T16:36:27.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:27 smithi112 ceph-mon[29122]: osdmap e73: 12 total, 12 up, 12 in 2024-04-25T16:36:27.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:27 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/403845828' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T16:36:27.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:27 smithi112 ceph-mon[29122]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T16:36:27.533 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:27 smithi112 ceph-mon[29122]: osdmap e74: 12 total, 12 up, 12 in 2024-04-25T16:36:27.535 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:36:27.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/1046119689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4102250 msgr2=0x7f5ff410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/1046119689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4102250 0x7f5ff410bcb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009ca0 tx=0x7f5fec02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/1046119689 shutdown_connections 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/1046119689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ff4101750 0x7f5ff4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/1046119689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4102250 0x7f5ff410bcb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/1046119689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ff41050a0 0x7f5ff41054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/1046119689 >> 172.21.15.112:0/1046119689 conn(0x7f5ff40fd370 msgr2=0x7f5ff40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/1046119689 shutdown_connections 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.644+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/1046119689 wait complete. 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 Processor -- start 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 -- start start 2024-04-25T16:36:27.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 0x7f5ff419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ff4102250 0x7f5ff419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ff41050a0 0x7f5ff41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410f630 con 0x7f5ff4101750 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410f4b0 con 0x7f5ff4102250 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ffa1ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff410f7b0 con 0x7f5ff41050a0 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 0x7f5ff419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 0x7f5ff419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52388/0 (socket says 172.21.15.112:52388) 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.645+0000 7f5ff37fe640 1 -- 172.21.15.112:0/226258321 learned_addr learned my addr 172.21.15.112:0/226258321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff2ffd640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ff4102250 0x7f5ff419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff3fff640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ff41050a0 0x7f5ff41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff37fe640 1 -- 172.21.15.112:0/226258321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ff41050a0 msgr2=0x7f5ff41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff37fe640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ff41050a0 0x7f5ff41a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff37fe640 1 -- 172.21.15.112:0/226258321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ff4102250 msgr2=0x7f5ff419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff37fe640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ff4102250 0x7f5ff419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff37fe640 1 -- 172.21.15.112:0/226258321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41a1cb0 con 0x7f5ff4101750 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff37fe640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 0x7f5ff419a170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e980 tx=0x7f5fe400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400cd80 con 0x7f5ff4101750 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe40108e0 con 0x7f5ff4101750 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41a1fa0 con 0x7f5ff4101750 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4018990 con 0x7f5ff4101750 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.646+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a2400 con 0x7f5ff4101750 2024-04-25T16:36:27.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.647+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff406a1c0 con 0x7f5ff4101750 2024-04-25T16:36:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.649+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f5fe401f020 con 0x7f5ff4101750 2024-04-25T16:36:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.650+0000 7f5ff0ff9640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.650+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7979+0+0 (secure 0 0 0) 0x7f5fe40df120 con 0x7f5ff4101750 2024-04-25T16:36:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.650+0000 7f5ff2ffd640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.650+0000 7f5ff2ffd640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ff419b210 tx=0x7f5fe800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:27.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.653+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ff406a1c0 con 0x7f5ff4101750 2024-04-25T16:36:27.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.831+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ff402dcb0 con 0x7f5ff4101750 2024-04-25T16:36:27.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.833+0000 7f5ff0ff9640 1 -- 172.21.15.112:0/226258321 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+22312 (secure 0 0 0) 0x7f5ff402dcb0 con 0x7f5ff4101750 2024-04-25T16:36:27.832 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:27.832 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":74,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:36:26.563240+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5fdc0afaf0 msgr2=0x7f5fdc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5ff419b210 tx=0x7f5fe800b390 comp rx=0 tx=0).stop 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 msgr2=0x7f5ff419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 0x7f5ff419a170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400e980 tx=0x7f5fe400ee40 comp rx=0 tx=0).stop 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 shutdown_connections 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5fdc0afaf0 0x7f5fdc0b2000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ff41050a0 0x7f5ff41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ff4102250 0x7f5ff419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 --2- 172.21.15.112:0/226258321 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ff4101750 0x7f5ff419a170 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 >> 172.21.15.112:0/226258321 conn(0x7f5ff40fd370 msgr2=0x7f5ff4107c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:27.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 shutdown_connections 2024-04-25T16:36:27.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.835+0000 7f5ffa1ee640 1 -- 172.21.15.112:0/226258321 wait complete. 2024-04-25T16:36:27.845 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 -- 172.21.15.112:0/825186135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674102a50 msgr2=0x7f4674103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 --2- 172.21.15.112:0/825186135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674102a50 0x7f4674103750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4664009900 tx=0x7f466402a410 comp rx=0 tx=0).stop 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 -- 172.21.15.112:0/825186135 shutdown_connections 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 --2- 172.21.15.112:0/825186135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4674103c90 0x7f46741040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 --2- 172.21.15.112:0/825186135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674108990 0x7f467410adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 --2- 172.21.15.112:0/825186135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674102a50 0x7f4674103750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 -- 172.21.15.112:0/825186135 >> 172.21.15.112:0/825186135 conn(0x7f46740fdc40 msgr2=0x7f46741000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 -- 172.21.15.112:0/825186135 shutdown_connections 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 -- 172.21.15.112:0/825186135 wait complete. 2024-04-25T16:36:27.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 Processor -- start 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.971+0000 7f467a986640 1 -- start start 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674102a50 0x7f4674199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 0x7f4674199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4674108990 0x7f46741a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f467410e4e0 con 0x7f4674103c90 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f467410e360 con 0x7f4674102a50 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f467410e660 con 0x7f4674108990 2024-04-25T16:36:27.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 0x7f4674199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 0x7f4674199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52418/0 (socket says 172.21.15.112:52418) 2024-04-25T16:36:27.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 -- 172.21.15.112:0/205923593 learned_addr learned my addr 172.21.15.112:0/205923593 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:27.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f4673fff640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674102a50 0x7f4674199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f4678f23640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4674108990 0x7f46741a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 -- 172.21.15.112:0/205923593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4674108990 msgr2=0x7f46741a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4674108990 0x7f46741a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 -- 172.21.15.112:0/205923593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674102a50 msgr2=0x7f4674199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674102a50 0x7f4674199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 -- 172.21.15.112:0/205923593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46741a0c90 con 0x7f4674103c90 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f4673fff640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674102a50 0x7f4674199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46737fe640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 0x7f4674199690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f466800d770 tx=0x7f466800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:27.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668017070 con 0x7f4674103c90 2024-04-25T16:36:27.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46680054d0 con 0x7f4674103c90 2024-04-25T16:36:27.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46741a0f80 con 0x7f4674103c90 2024-04-25T16:36:27.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668004c30 con 0x7f4674103c90 2024-04-25T16:36:27.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.972+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46741a1390 con 0x7f4674103c90 2024-04-25T16:36:27.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.973+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46741077a0 con 0x7f4674103c90 2024-04-25T16:36:27.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.975+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f4668004dd0 con 0x7f4674103c90 2024-04-25T16:36:27.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.975+0000 7f46717fa640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f465c0afaf0 0x7f465c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:27.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.976+0000 7f4673fff640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f465c0afaf0 0x7f465c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:27.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.976+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f46680dea40 con 0x7f4674103c90 2024-04-25T16:36:27.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.976+0000 7f4673fff640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f465c0afaf0 0x7f465c0b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4664009230 tx=0x7f4664000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:27.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:27.979+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46680a7770 con 0x7f4674103c90 2024-04-25T16:36:28.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.196+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f467402db50 con 0x7f4674103c90 2024-04-25T16:36:28.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.198+0000 7f46717fa640 1 -- 172.21.15.112:0/205923593 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f46680a7070 con 0x7f4674103c90 2024-04-25T16:36:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f465c0afaf0 msgr2=0x7f465c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f465c0afaf0 0x7f465c0b2000 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4664009230 tx=0x7f4664000bc0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 msgr2=0x7f4674199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 0x7f4674199690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f466800d770 tx=0x7f466800dc30 comp rx=0 tx=0).stop 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 shutdown_connections 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f465c0afaf0 0x7f465c0b2000 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4674108990 0x7f46741a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4674103c90 0x7f4674199690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 --2- 172.21.15.112:0/205923593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4674102a50 0x7f4674199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 >> 172.21.15.112:0/205923593 conn(0x7f46740fdc40 msgr2=0x7f4674104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 shutdown_connections 2024-04-25T16:36:28.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.201+0000 7f467a986640 1 -- 172.21.15.112:0/205923593 wait complete. 2024-04-25T16:36:28.213 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-25T16:36:28.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 -- 172.21.15.112:0/1118430726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 msgr2=0x7f2e281054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/1118430726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281054c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009900 tx=0x7f2e2402a410 comp rx=0 tx=0).stop 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 -- 172.21.15.112:0/1118430726 shutdown_connections 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/1118430726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e28101760 0x7f2e28101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/1118430726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e28102260 0x7f2e2810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/1118430726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281054c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 -- 172.21.15.112:0/1118430726 >> 172.21.15.112:0/1118430726 conn(0x7f2e280fd380 msgr2=0x7f2e280ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 -- 172.21.15.112:0/1118430726 shutdown_connections 2024-04-25T16:36:28.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.325+0000 7f2e2f71f640 1 -- 172.21.15.112:0/1118430726 wait complete. 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 Processor -- start 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 -- start start 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e28101760 0x7f2e2819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e28102260 0x7f2e2819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810f670 con 0x7f2e281050b0 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810f4f0 con 0x7f2e28101760 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2f71f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e2810f7f0 con 0x7f2e28102260 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2dcbc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2dcbc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52424/0 (socket says 172.21.15.112:52424) 2024-04-25T16:36:28.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.326+0000 7f2e2dcbc640 1 -- 172.21.15.112:0/433647169 learned_addr learned my addr 172.21.15.112:0/433647169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2ccba640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e28102260 0x7f2e2819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2d4bb640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e28101760 0x7f2e2819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2dcbc640 1 -- 172.21.15.112:0/433647169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e28102260 msgr2=0x7f2e2819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2dcbc640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e28102260 0x7f2e2819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2dcbc640 1 -- 172.21.15.112:0/433647169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e28101760 msgr2=0x7f2e2819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2dcbc640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e28101760 0x7f2e2819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2dcbc640 1 -- 172.21.15.112:0/433647169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e281a1e10 con 0x7f2e281050b0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2dcbc640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281a17a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000cad0 tx=0x7f2e2000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000ad60 con 0x7f2e281050b0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e20005020 con 0x7f2e281050b0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e281a2100 con 0x7f2e281050b0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000f6b0 con 0x7f2e281050b0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.327+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e281a2680 con 0x7f2e281050b0 2024-04-25T16:36:28.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.328+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e28108a40 con 0x7f2e281050b0 2024-04-25T16:36:28.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.330+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f2e200051d0 con 0x7f2e281050b0 2024-04-25T16:36:28.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.330+0000 7f2e1e7fc640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e140afaf0 0x7f2e140b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:28.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.330+0000 7f2e2d4bb640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e140afaf0 0x7f2e140b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.330+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f2e200def60 con 0x7f2e281050b0 2024-04-25T16:36:28.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.331+0000 7f2e2d4bb640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e140afaf0 0x7f2e140b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009230 tx=0x7f2e24031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:28.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.334+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e20014030 con 0x7f2e281050b0 2024-04-25T16:36:28.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: pgmap v280: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/403845828' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3248229939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/226258321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T16:36:28.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:28 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/205923593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T16:36:28.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:28.552+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f2e280637a0 con 0x7f2e281050b0 2024-04-25T16:36:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: pgmap v280: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/403845828' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T16:36:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T16:36:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T16:36:28.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3248229939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:28.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/226258321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:28.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T16:36:28.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:28 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/205923593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: pgmap v280: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/403845828' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3248229939' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/226258321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: osdmap e75: 12 total, 12 up, 12 in 2024-04-25T16:36:28.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:28 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/205923593' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T16:36:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.210+0000 7f2e1e7fc640 1 -- 172.21.15.112:0/433647169 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f2e200a7c90 con 0x7f2e281050b0 2024-04-25T16:36:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.213+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e140afaf0 msgr2=0x7f2e140b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.213+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e140afaf0 0x7f2e140b2000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009230 tx=0x7f2e24031400 comp rx=0 tx=0).stop 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.213+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 msgr2=0x7f2e281a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.213+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281a17a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000cad0 tx=0x7f2e2000ced0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 shutdown_connections 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e140afaf0 0x7f2e140b2000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e28102260 0x7f2e2819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e281050b0 0x7f2e281a17a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 --2- 172.21.15.112:0/433647169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e28101760 0x7f2e2819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 >> 172.21.15.112:0/433647169 conn(0x7f2e280fd380 msgr2=0x7f2e28105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:29.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 shutdown_connections 2024-04-25T16:36:29.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.214+0000 7f2e2f71f640 1 -- 172.21.15.112:0/433647169 wait complete. 2024-04-25T16:36:29.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 -- 172.21.15.112:0/2704536453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 msgr2=0x7ff144103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 --2- 172.21.15.112:0/2704536453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff144103700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff140009900 tx=0x7ff14002a410 comp rx=0 tx=0).stop 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 -- 172.21.15.112:0/2704536453 shutdown_connections 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 --2- 172.21.15.112:0/2704536453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff144103c40 0x7ff144104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 --2- 172.21.15.112:0/2704536453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 0x7ff14410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 --2- 172.21.15.112:0/2704536453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff144103700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 -- 172.21.15.112:0/2704536453 >> 172.21.15.112:0/2704536453 conn(0x7ff1440fdbf0 msgr2=0x7ff1441000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 -- 172.21.15.112:0/2704536453 shutdown_connections 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.343+0000 7ff14c708640 1 -- 172.21.15.112:0/2704536453 wait complete. 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 Processor -- start 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 -- start start 2024-04-25T16:36:29.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff1441991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff144103c40 0x7ff144199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 0x7ff1441a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410e4e0 con 0x7ff144102a00 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410e360 con 0x7ff144108940 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14c708640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410e660 con 0x7ff144103c40 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14a4a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff1441991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14a4a4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff1441991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52448/0 (socket says 172.21.15.112:52448) 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff14a4a4640 1 -- 172.21.15.112:0/3057255230 learned_addr learned my addr 172.21.15.112:0/3057255230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.344+0000 7ff149ca3640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff144103c40 0x7ff144199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14a4a4640 1 -- 172.21.15.112:0/3057255230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff144103c40 msgr2=0x7ff144199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14a4a4640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff144103c40 0x7ff144199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14aca5640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 0x7ff1441a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14a4a4640 1 -- 172.21.15.112:0/3057255230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 msgr2=0x7ff1441a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14a4a4640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 0x7ff1441a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14a4a4640 1 -- 172.21.15.112:0/3057255230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff144074e50 con 0x7ff144102a00 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14aca5640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 0x7ff1441a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14a4a4640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff1441991d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff140006010 tx=0x7ff140031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14003f030 con 0x7ff144102a00 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff140003aa0 con 0x7ff144102a00 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1440750e0 con 0x7ff144102a00 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140035070 con 0x7ff144102a00 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.345+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1440755d0 con 0x7ff144102a00 2024-04-25T16:36:29.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.346+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff144107750 con 0x7ff144102a00 2024-04-25T16:36:29.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.347+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7ff140042030 con 0x7ff144102a00 2024-04-25T16:36:29.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.348+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1300afaf0 0x7ff1300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:29.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.348+0000 7ff149ca3640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1300afaf0 0x7ff1300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:29.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.348+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff1400ff0b0 con 0x7ff144102a00 2024-04-25T16:36:29.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.348+0000 7ff149ca3640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1300afaf0 0x7ff1300b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff134005e10 tx=0x7ff134005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:29.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.353+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1400c7de0 con 0x7ff144102a00 2024-04-25T16:36:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/205923593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T16:36:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:29 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] up:active 2024-04-25T16:36:29.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:29 smithi116 ceph-mon[32712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T16:36:29.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/433647169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T16:36:29.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:29.570+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff14402db50 con 0x7ff144102a00 2024-04-25T16:36:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/205923593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T16:36:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:29 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] up:active 2024-04-25T16:36:29.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:29 smithi112 ceph-mon[29122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T16:36:29.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/433647169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T16:36:29.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/205923593' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T16:36:29.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:29 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] up:active 2024-04-25T16:36:29.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:29 smithi133 ceph-mon[32691]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T16:36:29.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/433647169' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T16:36:30.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.218+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3057255230 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7ff1400c76e0 con 0x7ff144102a00 2024-04-25T16:36:30.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1300afaf0 msgr2=0x7ff1300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1300afaf0 0x7ff1300b2000 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff134005e10 tx=0x7ff134005da0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 msgr2=0x7ff1441991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff1441991d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff140006010 tx=0x7ff140031a50 comp rx=0 tx=0).stop 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 shutdown_connections 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1300afaf0 0x7ff1300b2000 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff144103c40 0x7ff144199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff144108940 0x7ff1441a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 --2- 172.21.15.112:0/3057255230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff144102a00 0x7ff1441991d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 >> 172.21.15.112:0/3057255230 conn(0x7ff1440fdbf0 msgr2=0x7ff144104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 shutdown_connections 2024-04-25T16:36:30.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.221+0000 7ff14c708640 1 -- 172.21.15.112:0/3057255230 wait complete. 2024-04-25T16:36:30.235 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 -- 172.21.15.112:0/2770454526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6881050b0 msgr2=0x7fc6881054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/2770454526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6881050b0 0x7fc6881054c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc678009900 tx=0x7fc67802a410 comp rx=0 tx=0).stop 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 -- 172.21.15.112:0/2770454526 shutdown_connections 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/2770454526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc688101760 0x7fc688101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/2770454526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc688102260 0x7fc68810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/2770454526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6881050b0 0x7fc6881054c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 -- 172.21.15.112:0/2770454526 >> 172.21.15.112:0/2770454526 conn(0x7fc6880fd380 msgr2=0x7fc6880ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 -- 172.21.15.112:0/2770454526 shutdown_connections 2024-04-25T16:36:30.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.348+0000 7fc68dc6e640 1 -- 172.21.15.112:0/2770454526 wait complete. 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 Processor -- start 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 -- start start 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 0x7fc68819a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc688102260 0x7fc68819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6881050b0 0x7fc6881a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc68810f670 con 0x7fc688101760 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc68810f4f0 con 0x7fc6881050b0 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc68dc6e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc68810f7f0 con 0x7fc688102260 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 0x7fc68819a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 0x7fc68819a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52464/0 (socket says 172.21.15.112:52464) 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 -- 172.21.15.112:0/501882856 learned_addr learned my addr 172.21.15.112:0/501882856 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:30.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc687fff640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6881050b0 0x7fc6881a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc686ffd640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc688102260 0x7fc68819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 -- 172.21.15.112:0/501882856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc688102260 msgr2=0x7fc68819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc688102260 0x7fc68819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 -- 172.21.15.112:0/501882856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6881050b0 msgr2=0x7fc6881a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.349+0000 7fc6877fe640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6881050b0 0x7fc6881a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc6877fe640 1 -- 172.21.15.112:0/501882856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6881a1df0 con 0x7fc688101760 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc6877fe640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 0x7fc68819a2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc678006010 tx=0x7fc678004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67803f030 con 0x7fc688101760 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc678003aa0 con 0x7fc688101760 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6881a2080 con 0x7fc688101760 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678035070 con 0x7fc688101760 2024-04-25T16:36:30.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.350+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6881a25f0 con 0x7fc688101760 2024-04-25T16:36:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.351+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc688108a40 con 0x7fc688101760 2024-04-25T16:36:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.352+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fc678042030 con 0x7fc688101760 2024-04-25T16:36:30.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.353+0000 7fc684ff9640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6700afaf0 0x7fc6700b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:30.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.353+0000 7fc686ffd640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6700afaf0 0x7fc6700b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:30.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.353+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc6780ff5c0 con 0x7fc688101760 2024-04-25T16:36:30.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.353+0000 7fc686ffd640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6700afaf0 0x7fc6700b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc68819b350 tx=0x7fc67c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:30.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.357+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6780c82f0 con 0x7fc688101760 2024-04-25T16:36:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: pgmap v282: 97 pgs: 9 creating+peering, 54 active+clean, 34 unknown; 579 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T16:36:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T16:36:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/433647169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T16:36:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T16:36:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T16:36:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T16:36:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T16:36:30.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3057255230' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T16:36:30.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:30.575+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc68802db50 con 0x7fc688101760 2024-04-25T16:36:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: pgmap v282: 97 pgs: 9 creating+peering, 54 active+clean, 34 unknown; 579 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T16:36:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T16:36:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/433647169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T16:36:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T16:36:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T16:36:30.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T16:36:30.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T16:36:30.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:30 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3057255230' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T16:36:30.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: pgmap v282: 97 pgs: 9 creating+peering, 54 active+clean, 34 unknown; 579 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/433647169' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-25T16:36:30.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:30 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3057255230' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T16:36:31.008 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: 2024-04-25T16:36:30.732+0000 7f99f2434700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T16:36:31.008 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:36:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: 2024-04-25T16:36:30.732+0000 7f99f2434700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T16:36:31.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.230+0000 7fc684ff9640 1 -- 172.21.15.112:0/501882856 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fc6780c7bf0 con 0x7fc688101760 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6700afaf0 msgr2=0x7fc6700b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6700afaf0 0x7fc6700b2000 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc68819b350 tx=0x7fc67c008040 comp rx=0 tx=0).stop 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 msgr2=0x7fc68819a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 0x7fc68819a2b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc678006010 tx=0x7fc678004270 comp rx=0 tx=0).stop 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 shutdown_connections 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6700afaf0 0x7fc6700b2000 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc688102260 0x7fc68819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6881050b0 0x7fc6881a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 --2- 172.21.15.112:0/501882856 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc688101760 0x7fc68819a2b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 >> 172.21.15.112:0/501882856 conn(0x7fc6880fd380 msgr2=0x7fc688105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:31.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 shutdown_connections 2024-04-25T16:36:31.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.233+0000 7fc68dc6e640 1 -- 172.21.15.112:0/501882856 wait complete. 2024-04-25T16:36:31.246 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-25T16:36:31.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.360+0000 7f0806764640 1 -- 172.21.15.112:0/468621930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 msgr2=0x7f08001036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.360+0000 7f0806764640 1 --2- 172.21.15.112:0/468621930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f08001036f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f07f0009900 tx=0x7f07f002a410 comp rx=0 tx=0).stop 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 -- 172.21.15.112:0/468621930 shutdown_connections 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 --2- 172.21.15.112:0/468621930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0800108930 0x7f080010ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 --2- 172.21.15.112:0/468621930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0800103c30 0x7f0800104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 --2- 172.21.15.112:0/468621930 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f08001036f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 -- 172.21.15.112:0/468621930 >> 172.21.15.112:0/468621930 conn(0x7f08000fdbe0 msgr2=0x7f0800100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 -- 172.21.15.112:0/468621930 shutdown_connections 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 -- 172.21.15.112:0/468621930 wait complete. 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 Processor -- start 2024-04-25T16:36:31.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 -- start start 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.361+0000 7f0806764640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f080006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0800103c30 0x7f08001a1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0800108930 0x7f08001a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e4f0 con 0x7f08001029f0 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e370 con 0x7f0800103c30 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f080010e670 con 0x7f0800108930 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f080006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f080006bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52490/0 (socket says 172.21.15.112:52490) 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 -- 172.21.15.112:0/1082528710 learned_addr learned my addr 172.21.15.112:0/1082528710 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07ff7fe640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0800103c30 0x7f08001a1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0804d01640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0800108930 0x7f08001a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 -- 172.21.15.112:0/1082528710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0800108930 msgr2=0x7f08001a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0800108930 0x7f08001a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 -- 172.21.15.112:0/1082528710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0800103c30 msgr2=0x7f08001a1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0800103c30 0x7f08001a1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 -- 172.21.15.112:0/1082528710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001a3fd0 con 0x7f08001029f0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fffff640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f080006bd20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07f0006010 tx=0x7f07f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f003f030 con 0x7f08001029f0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f0031910 con 0x7f08001029f0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001a4260 con 0x7f08001029f0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0035070 con 0x7f08001029f0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.362+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a4750 con 0x7f08001029f0 2024-04-25T16:36:31.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.363+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0800107740 con 0x7f08001029f0 2024-04-25T16:36:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.365+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f07f0042030 con 0x7f08001029f0 2024-04-25T16:36:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.365+0000 7f07fd7fa640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f07e80afaf0 0x7f07e80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.365+0000 7f07ff7fe640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f07e80afaf0 0x7f07e80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.365+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f07f00ff260 con 0x7f08001029f0 2024-04-25T16:36:31.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.366+0000 7f07ff7fe640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f07e80afaf0 0x7f07e80b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f080006b6c0 tx=0x7f07f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:31.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.369+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07f00c7f90 con 0x7f08001029f0 2024-04-25T16:36:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3057255230' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T16:36:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] up:active 2024-04-25T16:36:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T16:36:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T16:36:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: Cluster is now healthy 2024-04-25T16:36:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T16:36:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T16:36:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T16:36:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:31 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/501882856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T16:36:31.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.574+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0800062fe0 con 0x7f07e80afaf0 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3057255230' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] up:active 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: Cluster is now healthy 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T16:36:31.579 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/501882856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T16:36:31.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.598+0000 7f07fd7fa640 1 -- 172.21.15.112:0/1082528710 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0800062fe0 con 0x7f07e80afaf0 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f07e80afaf0 msgr2=0x7f07e80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f07e80afaf0 0x7f07e80b2000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f080006b6c0 tx=0x7f07f4005f90 comp rx=0 tx=0).stop 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 msgr2=0x7f080006bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f080006bd20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f07f0006010 tx=0x7f07f0004270 comp rx=0 tx=0).stop 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 shutdown_connections 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f07e80afaf0 0x7f07e80b2000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0800108930 0x7f08001a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0800103c30 0x7f08001a1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 --2- 172.21.15.112:0/1082528710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08001029f0 0x7f080006bd20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 >> 172.21.15.112:0/1082528710 conn(0x7f08000fdbe0 msgr2=0x7f0800104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 shutdown_connections 2024-04-25T16:36:31.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.600+0000 7f0806764640 1 -- 172.21.15.112:0/1082528710 wait complete. 2024-04-25T16:36:31.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T16:36:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3057255230' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T16:36:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] up:active 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: Cluster is now healthy 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T16:36:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:31 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/501882856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T16:36:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 -- 172.21.15.112:0/3943996028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 msgr2=0x7f84bc1036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 --2- 172.21.15.112:0/3943996028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc1036e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009ca0 tx=0x7f84b402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 -- 172.21.15.112:0/3943996028 shutdown_connections 2024-04-25T16:36:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 --2- 172.21.15.112:0/3943996028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc108920 0x7f84bc10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 --2- 172.21.15.112:0/3943996028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84bc103c20 0x7f84bc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 --2- 172.21.15.112:0/3943996028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc1036e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.734+0000 7f84c3784640 1 -- 172.21.15.112:0/3943996028 >> 172.21.15.112:0/3943996028 conn(0x7f84bc0fdbd0 msgr2=0x7f84bc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 -- 172.21.15.112:0/3943996028 shutdown_connections 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 -- 172.21.15.112:0/3943996028 wait complete. 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 Processor -- start 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 -- start start 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84bc103c20 0x7f84bc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc108920 0x7f84bc1a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10e4e0 con 0x7f84bc1029e0 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10e360 con 0x7f84bc103c20 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c3784640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc10e660 con 0x7f84bc108920 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c1520640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.735+0000 7f84c1520640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52536/0 (socket says 172.21.15.112:52536) 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 -- 172.21.15.112:0/2440816884 learned_addr learned my addr 172.21.15.112:0/2440816884 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c0d1f640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84bc103c20 0x7f84bc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1d21640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc108920 0x7f84bc1a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 -- 172.21.15.112:0/2440816884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc108920 msgr2=0x7f84bc1a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc108920 0x7f84bc1a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 -- 172.21.15.112:0/2440816884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84bc103c20 msgr2=0x7f84bc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84bc103c20 0x7f84bc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 -- 172.21.15.112:0/2440816884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc1a1da0 con 0x7f84bc1029e0 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c1520640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc075860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004af0 tx=0x7f84b40316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4003b70 con 0x7f84bc1029e0 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc1a2030 con 0x7f84bc1029e0 2024-04-25T16:36:31.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b4003d10 con 0x7f84bc1029e0 2024-04-25T16:36:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a25d0 con 0x7f84bc1029e0 2024-04-25T16:36:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.736+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b402fa60 con 0x7f84bc1029e0 2024-04-25T16:36:31.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.737+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84bc06a160 con 0x7f84bc1029e0 2024-04-25T16:36:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.739+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f84b4004580 con 0x7f84bc1029e0 2024-04-25T16:36:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.739+0000 7f84b27fc640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f84a80afaf0 0x7f84a80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.739+0000 7f84c0d1f640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f84a80afaf0 0x7f84a80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.739+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f84b40fffe0 con 0x7f84bc1029e0 2024-04-25T16:36:31.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.740+0000 7f84c0d1f640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f84a80afaf0 0x7f84a80b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f84bc075200 tx=0x7f84ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:31.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.743+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84b40c8da0 con 0x7f84bc1029e0 2024-04-25T16:36:31.845 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:36:31.579+0000 7fb2a0f3c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:36:31.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:36:31.579+0000 7fb2a0f3c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:36:31.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:36:31.579+0000 7fb2a0f3c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:36:31.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:36:31.579+0000 7fb2a0f3c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:36:31.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:36:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:36:31.579+0000 7fb2a0f3c700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:36:31.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.946+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f84bc062fe0 con 0x7f84a80afaf0 2024-04-25T16:36:31.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.969+0000 7f84b27fc640 1 -- 172.21.15.112:0/2440816884 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f84bc062fe0 con 0x7f84a80afaf0 2024-04-25T16:36:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f84a80afaf0 msgr2=0x7f84a80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f84a80afaf0 0x7f84a80b2000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f84bc075200 tx=0x7f84ac005f90 comp rx=0 tx=0).stop 2024-04-25T16:36:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 msgr2=0x7f84bc075860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc075860 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f84b4004af0 tx=0x7f84b40316e0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 shutdown_connections 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f84a80afaf0 0x7f84a80b2000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84bc108920 0x7f84bc1a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84bc103c20 0x7f84bc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 --2- 172.21.15.112:0/2440816884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc1029e0 0x7f84bc075860 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 >> 172.21.15.112:0/2440816884 conn(0x7f84bc0fdbd0 msgr2=0x7f84bc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 shutdown_connections 2024-04-25T16:36:31.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:31.971+0000 7f84c3784640 1 -- 172.21.15.112:0/2440816884 wait complete. 2024-04-25T16:36:31.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T16:36:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 -- 172.21.15.112:0/3273915670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788103c40 msgr2=0x7fc788104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 --2- 172.21.15.112:0/3273915670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788103c40 0x7fc788104050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc778009900 tx=0x7fc77802a410 comp rx=0 tx=0).stop 2024-04-25T16:36:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 -- 172.21.15.112:0/3273915670 shutdown_connections 2024-04-25T16:36:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 --2- 172.21.15.112:0/3273915670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc788108940 0x7fc78810ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 --2- 172.21.15.112:0/3273915670 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788103c40 0x7fc788104050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 --2- 172.21.15.112:0/3273915670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc788102a00 0x7fc788103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 -- 172.21.15.112:0/3273915670 >> 172.21.15.112:0/3273915670 conn(0x7fc7880fdbf0 msgr2=0x7fc7881000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 -- 172.21.15.112:0/3273915670 shutdown_connections 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.093+0000 7fc7904f2640 1 -- 172.21.15.112:0/3273915670 wait complete. 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 Processor -- start 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 -- start start 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 0x7fc78810d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc788103c40 0x7fc78810b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc788108940 0x7fc78810beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810f630 con 0x7fc788102a00 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810f4b0 con 0x7fc788103c40 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc7904f2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810f7b0 con 0x7fc788108940 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 0x7fc78810d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 0x7fc78810d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52544/0 (socket says 172.21.15.112:52544) 2024-04-25T16:36:32.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 -- 172.21.15.112:0/195944079 learned_addr learned my addr 172.21.15.112:0/195944079 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78da8d640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc788103c40 0x7fc78810b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78ea8f640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc788108940 0x7fc78810beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 -- 172.21.15.112:0/195944079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc788108940 msgr2=0x7fc78810beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc788108940 0x7fc78810beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 -- 172.21.15.112:0/195944079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc788103c40 msgr2=0x7fc78810b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.094+0000 7fc78e28e640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc788103c40 0x7fc78810b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc78e28e640 1 -- 172.21.15.112:0/195944079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78810c5e0 con 0x7fc788102a00 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc78e28e640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 0x7fc78810d2d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc78400e9f0 tx=0x7fc78400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78400ccc0 con 0x7fc788102a00 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7881a42f0 con 0x7fc788102a00 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7840035f0 con 0x7fc788102a00 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78400ce60 con 0x7fc788102a00 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.095+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7881a47b0 con 0x7fc788102a00 2024-04-25T16:36:32.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.096+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc788107750 con 0x7fc788102a00 2024-04-25T16:36:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.097+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7fc784003aa0 con 0x7fc788102a00 2024-04-25T16:36:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.098+0000 7fc77f7fe640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc7740afaf0 0x7fc7740b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.098+0000 7fc78da8d640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc7740afaf0 0x7fc7740b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.098+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc7840def30 con 0x7fc788102a00 2024-04-25T16:36:32.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.098+0000 7fc78da8d640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc7740afaf0 0x7fc7740b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc778009230 tx=0x7fc778031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:32.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.101+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7840a7c60 con 0x7fc788102a00 2024-04-25T16:36:32.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:32 smithi112 ceph-mon[29122]: pgmap v283: 97 pgs: 90 active+clean, 7 unknown; 580 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 11 op/s 2024-04-25T16:36:32.230 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:32 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/501882856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T16:36:32.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.309+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc788062fe0 con 0x7fc7740afaf0 2024-04-25T16:36:32.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.314+0000 7fc77f7fe640 1 -- 172.21.15.112:0/195944079 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc788062fe0 con 0x7fc7740afaf0 2024-04-25T16:36:32.313 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_0/4c0cb907-a05a-4c22-86c8-2613854202fd 2024-04-25T16:36:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc7740afaf0 msgr2=0x7fc7740b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc7740afaf0 0x7fc7740b2000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc778009230 tx=0x7fc778031400 comp rx=0 tx=0).stop 2024-04-25T16:36:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 msgr2=0x7fc78810d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 0x7fc78810d2d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc78400e9f0 tx=0x7fc78400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 shutdown_connections 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc7740afaf0 0x7fc7740b2000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc788108940 0x7fc78810beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc788103c40 0x7fc78810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 --2- 172.21.15.112:0/195944079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc788102a00 0x7fc78810d2d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 >> 172.21.15.112:0/195944079 conn(0x7fc7880fdbf0 msgr2=0x7fc788104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 shutdown_connections 2024-04-25T16:36:32.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.316+0000 7fc7904f2640 1 -- 172.21.15.112:0/195944079 wait complete. 2024-04-25T16:36:32.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 -- 172.21.15.112:0/2763676163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec103c20 msgr2=0x7facec104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 --2- 172.21.15.112:0/2763676163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec103c20 0x7facec104030 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7facd8009900 tx=0x7facd802a410 comp rx=0 tx=0).stop 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 -- 172.21.15.112:0/2763676163 shutdown_connections 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 --2- 172.21.15.112:0/2763676163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec108920 0x7facec10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 --2- 172.21.15.112:0/2763676163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec103c20 0x7facec104030 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 --2- 172.21.15.112:0/2763676163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec1029e0 0x7facec1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 -- 172.21.15.112:0/2763676163 >> 172.21.15.112:0/2763676163 conn(0x7facec0fdbd0 msgr2=0x7facec100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:32.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 -- 172.21.15.112:0/2763676163 shutdown_connections 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.441+0000 7facf31c5640 1 -- 172.21.15.112:0/2763676163 wait complete. 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 Processor -- start 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 -- start start 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec1029e0 0x7facec19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103c20 0x7facec19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 0x7facec1a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7facec10f610 con 0x7facec108920 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7facec10f490 con 0x7facec1029e0 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf31c5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7facec10f790 con 0x7facec103c20 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf1762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 0x7facec1a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf1762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 0x7facec1a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52568/0 (socket says 172.21.15.112:52568) 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf1762640 1 -- 172.21.15.112:0/2737375135 learned_addr learned my addr 172.21.15.112:0/2737375135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.442+0000 7facf0f61640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec1029e0 0x7facec19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facebfff640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103c20 0x7facec19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf1762640 1 -- 172.21.15.112:0/2737375135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103c20 msgr2=0x7facec19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf1762640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103c20 0x7facec19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf1762640 1 -- 172.21.15.112:0/2737375135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec1029e0 msgr2=0x7facec19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf1762640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec1029e0 0x7facec19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf1762640 1 -- 172.21.15.112:0/2737375135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec1a2080 con 0x7facec108920 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facebfff640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103c20 0x7facec19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf1762640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 0x7facec1a1a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7face400eaa0 tx=0x7face400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face400cdb0 con 0x7facec108920 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec1a2370 con 0x7facec108920 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face4005020 con 0x7facec108920 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face40116d0 con 0x7facec108920 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.443+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec1a2870 con 0x7facec108920 2024-04-25T16:36:32.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.444+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facec107730 con 0x7facec108920 2024-04-25T16:36:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.446+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7face40051d0 con 0x7facec108920 2024-04-25T16:36:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.446+0000 7face9ffb640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7facdc0afaf0 0x7facdc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.446+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7face40deca0 con 0x7facec108920 2024-04-25T16:36:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.446+0000 7facf0f61640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7facdc0afaf0 0x7facdc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.447+0000 7facf0f61640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7facdc0afaf0 0x7facdc0b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7face0005fd0 tx=0x7face000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:32.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.451+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7face4016030 con 0x7facec108920 2024-04-25T16:36:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:32 smithi116 ceph-mon[32712]: pgmap v283: 97 pgs: 90 active+clean, 7 unknown; 580 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 11 op/s 2024-04-25T16:36:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:32 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/501882856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T16:36:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:32 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] up:active 2024-04-25T16:36:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:32 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T16:36:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:32 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:36:32.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:32 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] up:active 2024-04-25T16:36:32.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:32 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T16:36:32.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:32 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:36:32.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.662+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7facec02de30 con 0x7facdc0afaf0 2024-04-25T16:36:32.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.689+0000 7face9ffb640 1 -- 172.21.15.112:0/2737375135 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7facec02de30 con 0x7facdc0afaf0 2024-04-25T16:36:32.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7facdc0afaf0 msgr2=0x7facdc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7facdc0afaf0 0x7facdc0b2000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7face0005fd0 tx=0x7face000b410 comp rx=0 tx=0).stop 2024-04-25T16:36:32.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 msgr2=0x7facec1a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 0x7facec1a1a10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7face400eaa0 tx=0x7face400ef60 comp rx=0 tx=0).stop 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 shutdown_connections 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7facdc0afaf0 0x7facdc0b2000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7facec103c20 0x7facec19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7facec108920 0x7facec1a1a10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 --2- 172.21.15.112:0/2737375135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facec1029e0 0x7facec19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 >> 172.21.15.112:0/2737375135 conn(0x7facec0fdbd0 msgr2=0x7facec104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 shutdown_connections 2024-04-25T16:36:32.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.692+0000 7facf31c5640 1 -- 172.21.15.112:0/2737375135 wait complete. 2024-04-25T16:36:32.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:32 smithi133 ceph-mon[32691]: pgmap v283: 97 pgs: 90 active+clean, 7 unknown; 580 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 11 op/s 2024-04-25T16:36:32.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:32 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/501882856' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T16:36:32.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:32 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] up:active 2024-04-25T16:36:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:32 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-04-25T16:36:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:32 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:36:32.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 -- 172.21.15.112:0/1484682642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 msgr2=0x7f456810ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 --2- 172.21.15.112:0/1484682642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f456810ad50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f456002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 -- 172.21.15.112:0/1484682642 shutdown_connections 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 --2- 172.21.15.112:0/1484682642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f456810ad50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 --2- 172.21.15.112:0/1484682642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4568103c30 0x7f4568104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 --2- 172.21.15.112:0/1484682642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 0x7f45681036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 -- 172.21.15.112:0/1484682642 >> 172.21.15.112:0/1484682642 conn(0x7f45680fdc00 msgr2=0x7f4568100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 -- 172.21.15.112:0/1484682642 shutdown_connections 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.821+0000 7f45702a7640 1 -- 172.21.15.112:0/1484682642 wait complete. 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 Processor -- start 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 -- start start 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 0x7f456806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4568103c30 0x7f456819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f45681a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f456810e520 con 0x7f4568108930 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f456810e3a0 con 0x7f4568103c30 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f45702a7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f456810e6a0 con 0x7f45681029f0 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f45681a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f45681a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52590/0 (socket says 172.21.15.112:52590) 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 -- 172.21.15.112:0/610890953 learned_addr learned my addr 172.21.15.112:0/610890953 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:32.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e043640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 0x7f456806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456d842640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4568103c30 0x7f456819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 -- 172.21.15.112:0/610890953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 msgr2=0x7f456806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 0x7f456806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 -- 172.21.15.112:0/610890953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4568103c30 msgr2=0x7f456819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4568103c30 0x7f456819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e844640 1 -- 172.21.15.112:0/610890953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45681a1c50 con 0x7f4568108930 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.822+0000 7f456e043640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 0x7f456806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.823+0000 7f456e844640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f45681a14c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45600097b0 tx=0x7f4560003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.823+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560004240 con 0x7f4568108930 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.823+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456002f380 con 0x7f4568108930 2024-04-25T16:36:32.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.823+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681a1ee0 con 0x7f4568108930 2024-04-25T16:36:32.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.823+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456002fc30 con 0x7f4568108930 2024-04-25T16:36:32.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.823+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a2340 con 0x7f4568108930 2024-04-25T16:36:32.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.824+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45680692e0 con 0x7f4568108930 2024-04-25T16:36:32.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.825+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f45600043e0 con 0x7f4568108930 2024-04-25T16:36:32.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.826+0000 7f455f7fe640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f45540afaf0 0x7f45540b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:32.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.826+0000 7f456e043640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f45540afaf0 0x7f45540b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:32.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.826+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f4560100260 con 0x7f4568108930 2024-04-25T16:36:32.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.826+0000 7f456e043640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f45540afaf0 0x7f45540b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45681035f0 tx=0x7f456400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:32.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:32.831+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45600c8f90 con 0x7f4568108930 2024-04-25T16:36:33.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.043+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f4568063070 con 0x7f45540afaf0 2024-04-25T16:36:33.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.049+0000 7f455f7fe640 1 -- 172.21.15.112:0/610890953 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4568063070 con 0x7f45540afaf0 2024-04-25T16:36:33.048 INFO:teuthology.orchestra.run.smithi112.stdout:/volumes/qa/sv_1/d5283f8f-09b0-4e50-a257-9ae4d1f0c74a 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f45540afaf0 msgr2=0x7f45540b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f45540afaf0 0x7f45540b2000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f45681035f0 tx=0x7f456400a5e0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 msgr2=0x7f45681a14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f45681a14c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45600097b0 tx=0x7f4560003c20 comp rx=0 tx=0).stop 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 shutdown_connections 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4568108930 0x7f45681a14c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4568103c30 0x7f456819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f45540afaf0 0x7f45540b2000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 --2- 172.21.15.112:0/610890953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45681029f0 0x7f456806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 >> 172.21.15.112:0/610890953 conn(0x7f45680fdc00 msgr2=0x7f4568104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 shutdown_connections 2024-04-25T16:36:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.051+0000 7f45702a7640 1 -- 172.21.15.112:0/610890953 wait complete. 2024-04-25T16:36:33.061 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:36:33.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 -- 172.21.15.112:0/606611284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 msgr2=0x7f5038101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 --2- 172.21.15.112:0/606611284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f5038101ba0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f502802a410 comp rx=0 tx=0).stop 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 -- 172.21.15.112:0/606611284 shutdown_connections 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 --2- 172.21.15.112:0/606611284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5038102230 0x7f503810bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 --2- 172.21.15.112:0/606611284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f5038101ba0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 --2- 172.21.15.112:0/606611284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5038105080 0x7f5038105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 -- 172.21.15.112:0/606611284 >> 172.21.15.112:0/606611284 conn(0x7f50380fd330 msgr2=0x7f50380ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 -- 172.21.15.112:0/606611284 shutdown_connections 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 -- 172.21.15.112:0/606611284 wait complete. 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 Processor -- start 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.172+0000 7f503e31f640 1 -- start start 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f503819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5038102230 0x7f503819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5038105080 0x7f50381a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f503810f610 con 0x7f5038101730 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f503810f490 con 0x7f5038105080 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f503810f790 con 0x7f5038102230 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f503819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f503819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52616/0 (socket says 172.21.15.112:52616) 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 -- 172.21.15.112:0/917290863 learned_addr learned my addr 172.21.15.112:0/917290863 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f50377fe640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5038102230 0x7f503819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503c8bc640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5038105080 0x7f50381a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 -- 172.21.15.112:0/917290863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5038102230 msgr2=0x7f503819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5038102230 0x7f503819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 -- 172.21.15.112:0/917290863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5038105080 msgr2=0x7f50381a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5038105080 0x7f50381a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 -- 172.21.15.112:0/917290863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50381a1db0 con 0x7f5038101730 2024-04-25T16:36:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f5037fff640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f503819a270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f502c00e9f0 tx=0x7f502c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c00ccc0 con 0x7f5038101730 2024-04-25T16:36:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.173+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50381a20a0 con 0x7f5038101730 2024-04-25T16:36:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.174+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0035f0 con 0x7f5038101730 2024-04-25T16:36:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.174+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c00ce60 con 0x7f5038101730 2024-04-25T16:36:33.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.174+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a25f0 con 0x7f5038101730 2024-04-25T16:36:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.175+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5038108a10 con 0x7f5038101730 2024-04-25T16:36:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.176+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 16) ==== 147521+0+0 (secure 0 0 0) 0x7f502c003aa0 con 0x7f5038101730 2024-04-25T16:36:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.176+0000 7f50357fa640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f50200afaf0 0x7f50200b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.177+0000 7f50377fe640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f50200afaf0 0x7f50200b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.177+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f502c0def90 con 0x7f5038101730 2024-04-25T16:36:33.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.177+0000 7f50377fe640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f50200afaf0 0x7f50200b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5028000c00 tx=0x7f5028031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:33.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.181+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f502c0a7cc0 con 0x7f5038101730 2024-04-25T16:36:33.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.257+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f502c0a3850 con 0x7f5038101730 2024-04-25T16:36:33.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:33 smithi112 ceph-mon[29122]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:33.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:33 smithi112 ceph-mon[29122]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:33.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.408+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50380637a0 con 0x7f5038101730 2024-04-25T16:36:33.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.409+0000 7f50357fa640 1 -- 172.21.15.112:0/917290863 <== mon.0 v2:172.21.15.112:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11878 (secure 0 0 0) 0x7f502c082c00 con 0x7f5038101730 2024-04-25T16:36:33.407 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:33.408 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:36:31.235238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24514,24524,24560]},"id":1}]} 2024-04-25T16:36:33.408 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f50200afaf0 msgr2=0x7f50200b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f50200afaf0 0x7f50200b2000 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5028000c00 tx=0x7f5028031400 comp rx=0 tx=0).stop 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 msgr2=0x7f503819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f503819a270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f502c00e9f0 tx=0x7f502c00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 shutdown_connections 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f50200afaf0 0x7f50200b2000 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5038102230 0x7f503819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5038105080 0x7f50381a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 --2- 172.21.15.112:0/917290863 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5038101730 0x7f503819a270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 >> 172.21.15.112:0/917290863 conn(0x7f50380fd330 msgr2=0x7f5038105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 shutdown_connections 2024-04-25T16:36:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.411+0000 7f503e31f640 1 -- 172.21.15.112:0/917290863 wait complete. 2024-04-25T16:36:33.424 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:36:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:33 smithi116 ceph-mon[32712]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:33 smithi116 ceph-mon[32712]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 -- 172.21.15.112:0/2898831735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 msgr2=0x7feb1010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 --2- 172.21.15.112:0/2898831735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb1010ad40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb08009ca0 tx=0x7feb0802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 -- 172.21.15.112:0/2898831735 shutdown_connections 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 --2- 172.21.15.112:0/2898831735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 0x7feb10104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 --2- 172.21.15.112:0/2898831735 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb1010ad40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 --2- 172.21.15.112:0/2898831735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb101029e0 0x7feb101036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 -- 172.21.15.112:0/2898831735 >> 172.21.15.112:0/2898831735 conn(0x7feb100fdbd0 msgr2=0x7feb10100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:33.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 -- 172.21.15.112:0/2898831735 shutdown_connections 2024-04-25T16:36:33.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.535+0000 7feb162a8640 1 -- 172.21.15.112:0/2898831735 wait complete. 2024-04-25T16:36:33.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 Processor -- start 2024-04-25T16:36:33.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 -- start start 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb101029e0 0x7feb10075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 0x7feb1019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb101a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010e510 con 0x7feb10108920 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010e390 con 0x7feb101029e0 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb162a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feb1010e690 con 0x7feb10103c20 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb101a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb101a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52636/0 (socket says 172.21.15.112:52636) 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 -- 172.21.15.112:0/3822053486 learned_addr learned my addr 172.21.15.112:0/3822053486 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb0ffff640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb101029e0 0x7feb10075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb0f7fe640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 0x7feb1019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 -- 172.21.15.112:0/3822053486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 msgr2=0x7feb1019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 0x7feb1019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 -- 172.21.15.112:0/3822053486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb101029e0 msgr2=0x7feb10075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb101029e0 0x7feb10075860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb14845640 1 -- 172.21.15.112:0/3822053486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb101a1cd0 con 0x7feb10108920 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.536+0000 7feb0f7fe640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 0x7feb1019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.537+0000 7feb14845640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb101a14b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb080097b0 tx=0x7feb08003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.537+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb08004240 con 0x7feb10108920 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.537+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb101a1f60 con 0x7feb10108920 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.537+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0802f380 con 0x7feb10108920 2024-04-25T16:36:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.537+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0802fc30 con 0x7feb10108920 2024-04-25T16:36:33.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.537+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a23d0 con 0x7feb10108920 2024-04-25T16:36:33.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.538+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb10107730 con 0x7feb10108920 2024-04-25T16:36:33.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.539+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7feb080043e0 con 0x7feb10108920 2024-04-25T16:36:33.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.540+0000 7feb0d7fa640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7feaf80afb80 0x7feaf80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.540+0000 7feb0ffff640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7feaf80afb80 0x7feaf80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.540+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7feb081002c0 con 0x7feb10108920 2024-04-25T16:36:33.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.540+0000 7feb0ffff640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7feaf80afb80 0x7feaf80b2090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb101035e0 tx=0x7feb0400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:33.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.545+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb080c9000 con 0x7feb10108920 2024-04-25T16:36:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:33 smithi133 ceph-mon[32691]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:33 smithi133 ceph-mon[32691]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:33.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.737+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb1002dda0 con 0x7feb10108920 2024-04-25T16:36:33.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.739+0000 7feb0d7fa640 1 -- 172.21.15.112:0/3822053486 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22299 (secure 0 0 0) 0x7feb080098b0 con 0x7feb10108920 2024-04-25T16:36:33.738 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:33.738 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":75,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:36:27.955247+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7feaf80afb80 msgr2=0x7feaf80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7feaf80afb80 0x7feaf80b2090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feb101035e0 tx=0x7feb0400b390 comp rx=0 tx=0).stop 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 msgr2=0x7feb101a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb101a14b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb080097b0 tx=0x7feb08003c20 comp rx=0 tx=0).stop 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 shutdown_connections 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7feaf80afb80 0x7feaf80b2090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb10103c20 0x7feb1019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feb10108920 0x7feb101a14b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 --2- 172.21.15.112:0/3822053486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feb101029e0 0x7feb10075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 >> 172.21.15.112:0/3822053486 conn(0x7feb100fdbd0 msgr2=0x7feb10104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:33.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 shutdown_connections 2024-04-25T16:36:33.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.741+0000 7feb162a8640 1 -- 172.21.15.112:0/3822053486 wait complete. 2024-04-25T16:36:33.752 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T16:36:33.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:36:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.860+0000 7f3ebc26e640 1 -- 172.21.15.112:0/4237675188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 msgr2=0x7f3eb4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.860+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/4237675188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb4104010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009900 tx=0x7f3ea402a410 comp rx=0 tx=0).stop 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- 172.21.15.112:0/4237675188 shutdown_connections 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/4237675188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb4108900 0x7f3eb410ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/4237675188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb4104010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/4237675188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb41029c0 0x7f3eb41036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- 172.21.15.112:0/4237675188 >> 172.21.15.112:0/4237675188 conn(0x7f3eb40fdbd0 msgr2=0x7f3eb4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- 172.21.15.112:0/4237675188 shutdown_connections 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- 172.21.15.112:0/4237675188 wait complete. 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 Processor -- start 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- start start 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb41029c0 0x7f3eb419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb4108900 0x7f3eb41a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410f5f0 con 0x7f3eb4103c00 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410f470 con 0x7f3eb4108900 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3ebc26e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb410f770 con 0x7f3eb41029c0 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.861+0000 7f3eb9809640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52650/0 (socket says 172.21.15.112:52650) 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 -- 172.21.15.112:0/1068159980 learned_addr learned my addr 172.21.15.112:0/1068159980 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eba80b640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb4108900 0x7f3eb41a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eba00a640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb41029c0 0x7f3eb419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 -- 172.21.15.112:0/1068159980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb41029c0 msgr2=0x7f3eb419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb41029c0 0x7f3eb419a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 -- 172.21.15.112:0/1068159980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb4108900 msgr2=0x7f3eb41a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb4108900 0x7f3eb41a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 -- 172.21.15.112:0/1068159980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb41a1d60 con 0x7f3eb4103c00 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eb9809640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb419a760 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006010 tx=0x7f3ea4004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea403f030 con 0x7f3eb4103c00 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea4003cb0 con 0x7f3eb4103c00 2024-04-25T16:36:33.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb41a1ff0 con 0x7f3eb4103c00 2024-04-25T16:36:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4035070 con 0x7f3eb4103c00 2024-04-25T16:36:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.862+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb41a2400 con 0x7f3eb4103c00 2024-04-25T16:36:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.863+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb4107710 con 0x7f3eb4103c00 2024-04-25T16:36:33.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.865+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f3ea4042030 con 0x7f3eb4103c00 2024-04-25T16:36:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.865+0000 7f3eab7fe640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ea00afb80 0x7f3ea00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.865+0000 7f3eba00a640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ea00afb80 0x7f3ea00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.865+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f3ea40ff8d0 con 0x7f3eb4103c00 2024-04-25T16:36:33.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.866+0000 7f3eba00a640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ea00afb80 0x7f3ea00b2090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3eb41035c0 tx=0x7f3eb0009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:33.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:33.869+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ea4003e60 con 0x7f3eb4103c00 2024-04-25T16:36:34.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.041+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3eb4062fe0 con 0x7f3ea00afb80 2024-04-25T16:36:34.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.048+0000 7f3eab7fe640 1 -- 172.21.15.112:0/1068159980 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258868 (secure 0 0 0) 0x7f3eb4062fe0 con 0x7f3ea00afb80 2024-04-25T16:36:34.050 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:34.052 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T16:36:34.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ea00afb80 msgr2=0x7f3ea00b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ea00afb80 0x7f3ea00b2090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3eb41035c0 tx=0x7f3eb0009670 comp rx=0 tx=0).stop 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 msgr2=0x7f3eb419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb419a760 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4006010 tx=0x7f3ea4004340 comp rx=0 tx=0).stop 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 shutdown_connections 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb4108900 0x7f3eb41a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3eb4103c00 0x7f3eb419a760 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3ea00afb80 0x7f3ea00b2090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 --2- 172.21.15.112:0/1068159980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb41029c0 0x7f3eb419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 >> 172.21.15.112:0/1068159980 conn(0x7f3eb40fdbd0 msgr2=0x7f3eb41048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 shutdown_connections 2024-04-25T16:36:34.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.051+0000 7f3ebc26e640 1 -- 172.21.15.112:0/1068159980 wait complete. 2024-04-25T16:36:34.061 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-04-25T16:36:33.078947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595689,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":97,"num_read_kb":81,"num_write":220,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":175,"ondisk_log_size":175,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":346360,"kb_used_data":9640,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124595464,"statfs":{"total":1151940427776,"available":1151585755136,"internally_reserved":0,"allocated":9871360,"data_stored":6380910,"data_compressed":107133,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5321,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":97,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.886048"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725218+0000","last_change":"2024-04-25T16:36:26.988707+0000","last_active":"2024-04-25T16:36:28.725218+0000","last_peered":"2024-04-25T16:36:28.725218+0000","last_clean":"2024-04-25T16:36:28.725218+0000","last_became_active":"2024-04-25T16:36:26.988302+0000","last_became_peered":"2024-04-25T16:36:26.988302+0000","last_unstale":"2024-04-25T16:36:28.725218+0000","last_undegraded":"2024-04-25T16:36:28.725218+0000","last_fullsized":"2024-04-25T16:36:28.725218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:44.074090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.988824+0000","last_change":"2024-04-25T16:36:26.988147+0000","last_active":"2024-04-25T16:36:26.988824+0000","last_peered":"2024-04-25T16:36:26.988824+0000","last_clean":"2024-04-25T16:36:26.988824+0000","last_became_active":"2024-04-25T16:36:26.987695+0000","last_became_peered":"2024-04-25T16:36:26.987695+0000","last_unstale":"2024-04-25T16:36:26.988824+0000","last_undegraded":"2024-04-25T16:36:26.988824+0000","last_fullsized":"2024-04-25T16:36:26.988824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:16.338402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.988041+0000","last_change":"2024-04-25T16:36:26.986601+0000","last_active":"2024-04-25T16:36:26.988041+0000","last_peered":"2024-04-25T16:36:26.988041+0000","last_clean":"2024-04-25T16:36:26.988041+0000","last_became_active":"2024-04-25T16:36:26.985431+0000","last_became_peered":"2024-04-25T16:36:26.985431+0000","last_unstale":"2024-04-25T16:36:26.988041+0000","last_undegraded":"2024-04-25T16:36:26.988041+0000","last_fullsized":"2024-04-25T16:36:26.988041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:59.454059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229537+0000","last_change":"2024-04-25T16:36:26.985007+0000","last_active":"2024-04-25T16:36:29.229537+0000","last_peered":"2024-04-25T16:36:29.229537+0000","last_clean":"2024-04-25T16:36:29.229537+0000","last_became_active":"2024-04-25T16:36:26.983000+0000","last_became_peered":"2024-04-25T16:36:26.983000+0000","last_unstale":"2024-04-25T16:36:29.229537+0000","last_undegraded":"2024-04-25T16:36:29.229537+0000","last_fullsized":"2024-04-25T16:36:29.229537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:02.324409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959589+0000","last_change":"2024-04-25T16:36:26.970714+0000","last_active":"2024-04-25T16:36:27.959589+0000","last_peered":"2024-04-25T16:36:27.959589+0000","last_clean":"2024-04-25T16:36:27.959589+0000","last_became_active":"2024-04-25T16:36:26.970323+0000","last_became_peered":"2024-04-25T16:36:26.970323+0000","last_unstale":"2024-04-25T16:36:27.959589+0000","last_undegraded":"2024-04-25T16:36:27.959589+0000","last_fullsized":"2024-04-25T16:36:27.959589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:41.113178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.961889+0000","last_change":"2024-04-25T16:36:26.988388+0000","last_active":"2024-04-25T16:36:27.961889+0000","last_peered":"2024-04-25T16:36:27.961889+0000","last_clean":"2024-04-25T16:36:27.961889+0000","last_became_active":"2024-04-25T16:36:26.988090+0000","last_became_peered":"2024-04-25T16:36:26.988090+0000","last_unstale":"2024-04-25T16:36:27.961889+0000","last_undegraded":"2024-04-25T16:36:27.961889+0000","last_fullsized":"2024-04-25T16:36:27.961889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:19.666488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.975018+0000","last_change":"2024-04-25T16:36:26.971244+0000","last_active":"2024-04-25T16:36:26.975018+0000","last_peered":"2024-04-25T16:36:26.975018+0000","last_clean":"2024-04-25T16:36:26.975018+0000","last_became_active":"2024-04-25T16:36:26.970900+0000","last_became_peered":"2024-04-25T16:36:26.970900+0000","last_unstale":"2024-04-25T16:36:26.975018+0000","last_undegraded":"2024-04-25T16:36:26.975018+0000","last_fullsized":"2024-04-25T16:36:26.975018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:43.262385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200373+0000","last_change":"2024-04-25T16:36:26.982614+0000","last_active":"2024-04-25T16:36:29.200373+0000","last_peered":"2024-04-25T16:36:29.200373+0000","last_clean":"2024-04-25T16:36:29.200373+0000","last_became_active":"2024-04-25T16:36:26.981911+0000","last_became_peered":"2024-04-25T16:36:26.981911+0000","last_unstale":"2024-04-25T16:36:29.200373+0000","last_undegraded":"2024-04-25T16:36:29.200373+0000","last_fullsized":"2024-04-25T16:36:29.200373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:36.186614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229802+0000","last_change":"2024-04-25T16:36:26.972652+0000","last_active":"2024-04-25T16:36:29.229802+0000","last_peered":"2024-04-25T16:36:29.229802+0000","last_clean":"2024-04-25T16:36:29.229802+0000","last_became_active":"2024-04-25T16:36:26.972332+0000","last_became_peered":"2024-04-25T16:36:26.972332+0000","last_unstale":"2024-04-25T16:36:29.229802+0000","last_undegraded":"2024-04-25T16:36:29.229802+0000","last_fullsized":"2024-04-25T16:36:29.229802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:40.479663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962230+0000","last_change":"2024-04-25T16:36:26.989299+0000","last_active":"2024-04-25T16:36:27.962230+0000","last_peered":"2024-04-25T16:36:27.962230+0000","last_clean":"2024-04-25T16:36:27.962230+0000","last_became_active":"2024-04-25T16:36:26.988826+0000","last_became_peered":"2024-04-25T16:36:26.988826+0000","last_unstale":"2024-04-25T16:36:27.962230+0000","last_undegraded":"2024-04-25T16:36:27.962230+0000","last_fullsized":"2024-04-25T16:36:27.962230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:40.549866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.977705+0000","last_change":"2024-04-25T16:36:26.971847+0000","last_active":"2024-04-25T16:36:26.977705+0000","last_peered":"2024-04-25T16:36:26.977705+0000","last_clean":"2024-04-25T16:36:26.977705+0000","last_became_active":"2024-04-25T16:36:26.971417+0000","last_became_peered":"2024-04-25T16:36:26.971417+0000","last_unstale":"2024-04-25T16:36:26.977705+0000","last_undegraded":"2024-04-25T16:36:26.977705+0000","last_fullsized":"2024-04-25T16:36:26.977705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:03.870584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229735+0000","last_change":"2024-04-25T16:36:26.985714+0000","last_active":"2024-04-25T16:36:29.229735+0000","last_peered":"2024-04-25T16:36:29.229735+0000","last_clean":"2024-04-25T16:36:29.229735+0000","last_became_active":"2024-04-25T16:36:26.985402+0000","last_became_peered":"2024-04-25T16:36:26.985402+0000","last_unstale":"2024-04-25T16:36:29.229735+0000","last_undegraded":"2024-04-25T16:36:29.229735+0000","last_fullsized":"2024-04-25T16:36:29.229735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:32.953895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.979214+0000","last_change":"2024-04-25T16:36:26.976917+0000","last_active":"2024-04-25T16:36:26.979214+0000","last_peered":"2024-04-25T16:36:26.979214+0000","last_clean":"2024-04-25T16:36:26.979214+0000","last_became_active":"2024-04-25T16:36:26.976452+0000","last_became_peered":"2024-04-25T16:36:26.976452+0000","last_unstale":"2024-04-25T16:36:26.979214+0000","last_undegraded":"2024-04-25T16:36:26.979214+0000","last_fullsized":"2024-04-25T16:36:26.979214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:53.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.988461+0000","last_change":"2024-04-25T16:36:26.987433+0000","last_active":"2024-04-25T16:36:26.988461+0000","last_peered":"2024-04-25T16:36:26.988461+0000","last_clean":"2024-04-25T16:36:26.988461+0000","last_became_active":"2024-04-25T16:36:26.987106+0000","last_became_peered":"2024-04-25T16:36:26.987106+0000","last_unstale":"2024-04-25T16:36:26.988461+0000","last_undegraded":"2024-04-25T16:36:26.988461+0000","last_fullsized":"2024-04-25T16:36:26.988461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:57.671061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.986500+0000","last_change":"2024-04-25T16:36:26.984530+0000","last_active":"2024-04-25T16:36:26.986500+0000","last_peered":"2024-04-25T16:36:26.986500+0000","last_clean":"2024-04-25T16:36:26.986500+0000","last_became_active":"2024-04-25T16:36:26.984220+0000","last_became_peered":"2024-04-25T16:36:26.984220+0000","last_unstale":"2024-04-25T16:36:26.986500+0000","last_undegraded":"2024-04-25T16:36:26.986500+0000","last_fullsized":"2024-04-25T16:36:26.986500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:40.810663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.227536+0000","last_change":"2024-04-25T16:36:26.980249+0000","last_active":"2024-04-25T16:36:29.227536+0000","last_peered":"2024-04-25T16:36:29.227536+0000","last_clean":"2024-04-25T16:36:29.227536+0000","last_became_active":"2024-04-25T16:36:26.979861+0000","last_became_peered":"2024-04-25T16:36:26.979861+0000","last_unstale":"2024-04-25T16:36:29.227536+0000","last_undegraded":"2024-04-25T16:36:29.227536+0000","last_fullsized":"2024-04-25T16:36:29.227536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:26.753789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725316+0000","last_change":"2024-04-25T16:36:26.988022+0000","last_active":"2024-04-25T16:36:28.725316+0000","last_peered":"2024-04-25T16:36:28.725316+0000","last_clean":"2024-04-25T16:36:28.725316+0000","last_became_active":"2024-04-25T16:36:26.987609+0000","last_became_peered":"2024-04-25T16:36:26.987609+0000","last_unstale":"2024-04-25T16:36:28.725316+0000","last_undegraded":"2024-04-25T16:36:28.725316+0000","last_fullsized":"2024-04-25T16:36:28.725316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.819517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.958932+0000","last_change":"2024-04-25T16:36:26.978511+0000","last_active":"2024-04-25T16:36:27.958932+0000","last_peered":"2024-04-25T16:36:27.958932+0000","last_clean":"2024-04-25T16:36:27.958932+0000","last_became_active":"2024-04-25T16:36:26.978180+0000","last_became_peered":"2024-04-25T16:36:26.978180+0000","last_unstale":"2024-04-25T16:36:27.958932+0000","last_undegraded":"2024-04-25T16:36:27.958932+0000","last_fullsized":"2024-04-25T16:36:27.958932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:58.931649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.967205+0000","last_change":"2024-04-25T16:36:26.979313+0000","last_active":"2024-04-25T16:36:27.967205+0000","last_peered":"2024-04-25T16:36:27.967205+0000","last_clean":"2024-04-25T16:36:27.967205+0000","last_became_active":"2024-04-25T16:36:26.978958+0000","last_became_peered":"2024-04-25T16:36:26.978958+0000","last_unstale":"2024-04-25T16:36:27.967205+0000","last_undegraded":"2024-04-25T16:36:27.967205+0000","last_fullsized":"2024-04-25T16:36:27.967205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:57.379459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.236580+0000","last_change":"2024-04-25T16:36:26.216356+0000","last_active":"2024-04-25T16:36:30.236580+0000","last_peered":"2024-04-25T16:36:30.236580+0000","last_clean":"2024-04-25T16:36:30.236580+0000","last_became_active":"2024-04-25T16:36:26.215935+0000","last_became_peered":"2024-04-25T16:36:26.215935+0000","last_unstale":"2024-04-25T16:36:30.236580+0000","last_undegraded":"2024-04-25T16:36:30.236580+0000","last_fullsized":"2024-04-25T16:36:30.236580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:58.186003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200895+0000","last_change":"2024-04-25T16:36:26.981307+0000","last_active":"2024-04-25T16:36:29.200895+0000","last_peered":"2024-04-25T16:36:29.200895+0000","last_clean":"2024-04-25T16:36:29.200895+0000","last_became_active":"2024-04-25T16:36:26.980973+0000","last_became_peered":"2024-04-25T16:36:26.980973+0000","last_unstale":"2024-04-25T16:36:29.200895+0000","last_undegraded":"2024-04-25T16:36:29.200895+0000","last_fullsized":"2024-04-25T16:36:29.200895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:45.469266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.201000+0000","last_change":"2024-04-25T16:36:26.218839+0000","last_active":"2024-04-25T16:36:29.201000+0000","last_peered":"2024-04-25T16:36:29.201000+0000","last_clean":"2024-04-25T16:36:29.201000+0000","last_became_active":"2024-04-25T16:36:26.218487+0000","last_became_peered":"2024-04-25T16:36:26.218487+0000","last_unstale":"2024-04-25T16:36:29.201000+0000","last_undegraded":"2024-04-25T16:36:29.201000+0000","last_fullsized":"2024-04-25T16:36:29.201000+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:05.164084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.981712+0000","last_change":"2024-04-25T16:36:26.977277+0000","last_active":"2024-04-25T16:36:26.981712+0000","last_peered":"2024-04-25T16:36:26.981712+0000","last_clean":"2024-04-25T16:36:26.981712+0000","last_became_active":"2024-04-25T16:36:26.974805+0000","last_became_peered":"2024-04-25T16:36:26.974805+0000","last_unstale":"2024-04-25T16:36:26.981712+0000","last_undegraded":"2024-04-25T16:36:26.981712+0000","last_fullsized":"2024-04-25T16:36:26.981712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:25.219827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959883+0000","last_change":"2024-04-25T16:36:26.215757+0000","last_active":"2024-04-25T16:36:27.959883+0000","last_peered":"2024-04-25T16:36:27.959883+0000","last_clean":"2024-04-25T16:36:27.959883+0000","last_became_active":"2024-04-25T16:36:26.215424+0000","last_became_peered":"2024-04-25T16:36:26.215424+0000","last_unstale":"2024-04-25T16:36:27.959883+0000","last_undegraded":"2024-04-25T16:36:27.959883+0000","last_fullsized":"2024-04-25T16:36:27.959883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:23.094599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.228561+0000","last_change":"2024-04-25T16:36:26.978552+0000","last_active":"2024-04-25T16:36:29.228561+0000","last_peered":"2024-04-25T16:36:29.228561+0000","last_clean":"2024-04-25T16:36:29.228561+0000","last_became_active":"2024-04-25T16:36:26.978185+0000","last_became_peered":"2024-04-25T16:36:26.978185+0000","last_unstale":"2024-04-25T16:36:29.228561+0000","last_undegraded":"2024-04-25T16:36:29.228561+0000","last_fullsized":"2024-04-25T16:36:29.228561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:19.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959699+0000","last_change":"2024-04-25T16:36:26.207361+0000","last_active":"2024-04-25T16:36:27.959699+0000","last_peered":"2024-04-25T16:36:27.959699+0000","last_clean":"2024-04-25T16:36:27.959699+0000","last_became_active":"2024-04-25T16:36:26.206968+0000","last_became_peered":"2024-04-25T16:36:26.206968+0000","last_unstale":"2024-04-25T16:36:27.959699+0000","last_undegraded":"2024-04-25T16:36:27.959699+0000","last_fullsized":"2024-04-25T16:36:27.959699+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:54.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448323+0000","last_change":"2024-04-25T16:36:26.979723+0000","last_active":"2024-04-25T16:36:28.448323+0000","last_peered":"2024-04-25T16:36:28.448323+0000","last_clean":"2024-04-25T16:36:28.448323+0000","last_became_active":"2024-04-25T16:36:26.979377+0000","last_became_peered":"2024-04-25T16:36:26.979377+0000","last_unstale":"2024-04-25T16:36:28.448323+0000","last_undegraded":"2024-04-25T16:36:28.448323+0000","last_fullsized":"2024-04-25T16:36:28.448323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:50.312759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:32.248765+0000","last_change":"2024-04-25T16:36:26.217074+0000","last_active":"2024-04-25T16:36:32.248765+0000","last_peered":"2024-04-25T16:36:32.248765+0000","last_clean":"2024-04-25T16:36:32.248765+0000","last_became_active":"2024-04-25T16:36:26.216686+0000","last_became_peered":"2024-04-25T16:36:26.216686+0000","last_unstale":"2024-04-25T16:36:32.248765+0000","last_undegraded":"2024-04-25T16:36:32.248765+0000","last_fullsized":"2024-04-25T16:36:32.248765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:35.780179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959440+0000","last_change":"2024-04-25T16:36:26.972190+0000","last_active":"2024-04-25T16:36:27.959440+0000","last_peered":"2024-04-25T16:36:27.959440+0000","last_clean":"2024-04-25T16:36:27.959440+0000","last_became_active":"2024-04-25T16:36:26.971630+0000","last_became_peered":"2024-04-25T16:36:26.971630+0000","last_unstale":"2024-04-25T16:36:27.959440+0000","last_undegraded":"2024-04-25T16:36:27.959440+0000","last_fullsized":"2024-04-25T16:36:27.959440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:00.009040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.961802+0000","last_change":"2024-04-25T16:36:26.211703+0000","last_active":"2024-04-25T16:36:27.961802+0000","last_peered":"2024-04-25T16:36:27.961802+0000","last_clean":"2024-04-25T16:36:27.961802+0000","last_became_active":"2024-04-25T16:36:26.211322+0000","last_became_peered":"2024-04-25T16:36:26.211322+0000","last_unstale":"2024-04-25T16:36:27.961802+0000","last_undegraded":"2024-04-25T16:36:27.961802+0000","last_fullsized":"2024-04-25T16:36:27.961802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:15.353208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448274+0000","last_change":"2024-04-25T16:36:26.982502+0000","last_active":"2024-04-25T16:36:28.448274+0000","last_peered":"2024-04-25T16:36:28.448274+0000","last_clean":"2024-04-25T16:36:28.448274+0000","last_became_active":"2024-04-25T16:36:26.982112+0000","last_became_peered":"2024-04-25T16:36:26.982112+0000","last_unstale":"2024-04-25T16:36:28.448274+0000","last_undegraded":"2024-04-25T16:36:28.448274+0000","last_fullsized":"2024-04-25T16:36:28.448274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:54.106473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.962201+0000","last_change":"2024-04-25T16:36:26.217225+0000","last_active":"2024-04-25T16:36:26.962201+0000","last_peered":"2024-04-25T16:36:26.962201+0000","last_clean":"2024-04-25T16:36:26.962201+0000","last_became_active":"2024-04-25T16:36:26.216777+0000","last_became_peered":"2024-04-25T16:36:26.216777+0000","last_unstale":"2024-04-25T16:36:26.962201+0000","last_undegraded":"2024-04-25T16:36:26.962201+0000","last_fullsized":"2024-04-25T16:36:26.962201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:21.433977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200459+0000","last_change":"2024-04-25T16:36:26.970197+0000","last_active":"2024-04-25T16:36:29.200459+0000","last_peered":"2024-04-25T16:36:29.200459+0000","last_clean":"2024-04-25T16:36:29.200459+0000","last_became_active":"2024-04-25T16:36:26.969844+0000","last_became_peered":"2024-04-25T16:36:26.969844+0000","last_unstale":"2024-04-25T16:36:29.200459+0000","last_undegraded":"2024-04-25T16:36:29.200459+0000","last_fullsized":"2024-04-25T16:36:29.200459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:35.397143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.447934+0000","last_change":"2024-04-25T16:36:26.214731+0000","last_active":"2024-04-25T16:36:28.447934+0000","last_peered":"2024-04-25T16:36:28.447934+0000","last_clean":"2024-04-25T16:36:28.447934+0000","last_became_active":"2024-04-25T16:36:26.214343+0000","last_became_peered":"2024-04-25T16:36:26.214343+0000","last_unstale":"2024-04-25T16:36:28.447934+0000","last_undegraded":"2024-04-25T16:36:28.447934+0000","last_fullsized":"2024-04-25T16:36:28.447934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:08:10.156665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.958719+0000","last_change":"2024-04-25T16:36:26.981458+0000","last_active":"2024-04-25T16:36:27.958719+0000","last_peered":"2024-04-25T16:36:27.958719+0000","last_clean":"2024-04-25T16:36:27.958719+0000","last_became_active":"2024-04-25T16:36:26.981116+0000","last_became_peered":"2024-04-25T16:36:26.981116+0000","last_unstale":"2024-04-25T16:36:27.958719+0000","last_undegraded":"2024-04-25T16:36:27.958719+0000","last_fullsized":"2024-04-25T16:36:27.958719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:20.549670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962157+0000","last_change":"2024-04-25T16:36:26.218690+0000","last_active":"2024-04-25T16:36:27.962157+0000","last_peered":"2024-04-25T16:36:27.962157+0000","last_clean":"2024-04-25T16:36:27.962157+0000","last_became_active":"2024-04-25T16:36:26.218287+0000","last_became_peered":"2024-04-25T16:36:26.218287+0000","last_unstale":"2024-04-25T16:36:27.962157+0000","last_undegraded":"2024-04-25T16:36:27.962157+0000","last_fullsized":"2024-04-25T16:36:27.962157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:42.394413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.230451+0000","last_change":"2024-04-25T16:36:26.980356+0000","last_active":"2024-04-25T16:36:29.230451+0000","last_peered":"2024-04-25T16:36:29.230451+0000","last_clean":"2024-04-25T16:36:29.230451+0000","last_became_active":"2024-04-25T16:36:26.980005+0000","last_became_peered":"2024-04-25T16:36:26.980005+0000","last_unstale":"2024-04-25T16:36:29.230451+0000","last_undegraded":"2024-04-25T16:36:29.230451+0000","last_fullsized":"2024-04-25T16:36:29.230451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:37.263655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.959445+0000","last_change":"2024-04-25T16:36:26.217818+0000","last_active":"2024-04-25T16:36:26.959445+0000","last_peered":"2024-04-25T16:36:26.959445+0000","last_clean":"2024-04-25T16:36:26.959445+0000","last_became_active":"2024-04-25T16:36:26.217317+0000","last_became_peered":"2024-04-25T16:36:26.217317+0000","last_unstale":"2024-04-25T16:36:26.959445+0000","last_undegraded":"2024-04-25T16:36:26.959445+0000","last_fullsized":"2024-04-25T16:36:26.959445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:29.985735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.987652+0000","last_change":"2024-04-25T16:36:26.985901+0000","last_active":"2024-04-25T16:36:26.987652+0000","last_peered":"2024-04-25T16:36:26.987652+0000","last_clean":"2024-04-25T16:36:26.987652+0000","last_became_active":"2024-04-25T16:36:26.983463+0000","last_became_peered":"2024-04-25T16:36:26.983463+0000","last_unstale":"2024-04-25T16:36:26.987652+0000","last_undegraded":"2024-04-25T16:36:26.987652+0000","last_fullsized":"2024-04-25T16:36:26.987652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:40.246613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.231209+0000","last_change":"2024-04-25T16:36:26.208756+0000","last_active":"2024-04-25T16:36:29.231209+0000","last_peered":"2024-04-25T16:36:29.231209+0000","last_clean":"2024-04-25T16:36:29.231209+0000","last_became_active":"2024-04-25T16:36:26.208384+0000","last_became_peered":"2024-04-25T16:36:26.208384+0000","last_unstale":"2024-04-25T16:36:29.231209+0000","last_undegraded":"2024-04-25T16:36:29.231209+0000","last_fullsized":"2024-04-25T16:36:29.231209+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.849402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.228445+0000","last_change":"2024-04-25T16:36:26.981611+0000","last_active":"2024-04-25T16:36:29.228445+0000","last_peered":"2024-04-25T16:36:29.228445+0000","last_clean":"2024-04-25T16:36:29.228445+0000","last_became_active":"2024-04-25T16:36:26.981311+0000","last_became_peered":"2024-04-25T16:36:26.981311+0000","last_unstale":"2024-04-25T16:36:29.228445+0000","last_undegraded":"2024-04-25T16:36:29.228445+0000","last_fullsized":"2024-04-25T16:36:29.228445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:05.360035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962222+0000","last_change":"2024-04-25T16:36:26.214215+0000","last_active":"2024-04-25T16:36:27.962222+0000","last_peered":"2024-04-25T16:36:27.962222+0000","last_clean":"2024-04-25T16:36:27.962222+0000","last_became_active":"2024-04-25T16:36:26.213848+0000","last_became_peered":"2024-04-25T16:36:26.213848+0000","last_unstale":"2024-04-25T16:36:27.962222+0000","last_undegraded":"2024-04-25T16:36:27.962222+0000","last_fullsized":"2024-04-25T16:36:27.962222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:32.589646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959796+0000","last_change":"2024-04-25T16:36:26.976087+0000","last_active":"2024-04-25T16:36:27.959796+0000","last_peered":"2024-04-25T16:36:27.959796+0000","last_clean":"2024-04-25T16:36:27.959796+0000","last_became_active":"2024-04-25T16:36:26.975726+0000","last_became_peered":"2024-04-25T16:36:26.975726+0000","last_unstale":"2024-04-25T16:36:27.959796+0000","last_undegraded":"2024-04-25T16:36:27.959796+0000","last_fullsized":"2024-04-25T16:36:27.959796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:02.668930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.232489+0000","last_change":"2024-04-25T16:36:26.208815+0000","last_active":"2024-04-25T16:36:30.232489+0000","last_peered":"2024-04-25T16:36:30.232489+0000","last_clean":"2024-04-25T16:36:30.232489+0000","last_became_active":"2024-04-25T16:36:26.208485+0000","last_became_peered":"2024-04-25T16:36:26.208485+0000","last_unstale":"2024-04-25T16:36:30.232489+0000","last_undegraded":"2024-04-25T16:36:30.232489+0000","last_fullsized":"2024-04-25T16:36:30.232489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:11.984900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.976692+0000","last_change":"2024-04-25T16:36:26.975567+0000","last_active":"2024-04-25T16:36:26.976692+0000","last_peered":"2024-04-25T16:36:26.976692+0000","last_clean":"2024-04-25T16:36:26.976692+0000","last_became_active":"2024-04-25T16:36:26.975212+0000","last_became_peered":"2024-04-25T16:36:26.975212+0000","last_unstale":"2024-04-25T16:36:26.976692+0000","last_undegraded":"2024-04-25T16:36:26.976692+0000","last_fullsized":"2024-04-25T16:36:26.976692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:09.010964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.233052+0000","last_change":"2024-04-25T16:36:26.214765+0000","last_active":"2024-04-25T16:36:29.233052+0000","last_peered":"2024-04-25T16:36:29.233052+0000","last_clean":"2024-04-25T16:36:29.233052+0000","last_became_active":"2024-04-25T16:36:26.214378+0000","last_became_peered":"2024-04-25T16:36:26.214378+0000","last_unstale":"2024-04-25T16:36:29.233052+0000","last_undegraded":"2024-04-25T16:36:29.233052+0000","last_fullsized":"2024-04-25T16:36:29.233052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:00.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.981932+0000","last_change":"2024-04-25T16:36:26.978074+0000","last_active":"2024-04-25T16:36:26.981932+0000","last_peered":"2024-04-25T16:36:26.981932+0000","last_clean":"2024-04-25T16:36:26.981932+0000","last_became_active":"2024-04-25T16:36:26.977706+0000","last_became_peered":"2024-04-25T16:36:26.977706+0000","last_unstale":"2024-04-25T16:36:26.981932+0000","last_undegraded":"2024-04-25T16:36:26.981932+0000","last_fullsized":"2024-04-25T16:36:26.981932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.591511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:31.248003+0000","last_change":"2024-04-25T16:36:26.211526+0000","last_active":"2024-04-25T16:36:31.248003+0000","last_peered":"2024-04-25T16:36:31.248003+0000","last_clean":"2024-04-25T16:36:31.248003+0000","last_became_active":"2024-04-25T16:36:26.211134+0000","last_became_peered":"2024-04-25T16:36:26.211134+0000","last_unstale":"2024-04-25T16:36:31.248003+0000","last_undegraded":"2024-04-25T16:36:31.248003+0000","last_fullsized":"2024-04-25T16:36:31.248003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:25:39.743843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.981989+0000","last_change":"2024-04-25T16:36:26.979766+0000","last_active":"2024-04-25T16:36:26.981989+0000","last_peered":"2024-04-25T16:36:26.981989+0000","last_clean":"2024-04-25T16:36:26.981989+0000","last_became_active":"2024-04-25T16:36:26.979460+0000","last_became_peered":"2024-04-25T16:36:26.979460+0000","last_unstale":"2024-04-25T16:36:26.981989+0000","last_undegraded":"2024-04-25T16:36:26.981989+0000","last_fullsized":"2024-04-25T16:36:26.981989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:21.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'10","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.228207+0000","last_change":"2024-04-25T16:36:26.212112+0000","last_active":"2024-04-25T16:36:29.228207+0000","last_peered":"2024-04-25T16:36:29.228207+0000","last_clean":"2024-04-25T16:36:29.228207+0000","last_became_active":"2024-04-25T16:36:26.211746+0000","last_became_peered":"2024-04-25T16:36:26.211746+0000","last_unstale":"2024-04-25T16:36:29.228207+0000","last_undegraded":"2024-04-25T16:36:29.228207+0000","last_fullsized":"2024-04-25T16:36:29.228207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:24.682525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448052+0000","last_change":"2024-04-25T16:36:26.980275+0000","last_active":"2024-04-25T16:36:28.448052+0000","last_peered":"2024-04-25T16:36:28.448052+0000","last_clean":"2024-04-25T16:36:28.448052+0000","last_became_active":"2024-04-25T16:36:26.979912+0000","last_became_peered":"2024-04-25T16:36:26.979912+0000","last_unstale":"2024-04-25T16:36:28.448052+0000","last_undegraded":"2024-04-25T16:36:28.448052+0000","last_fullsized":"2024-04-25T16:36:28.448052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:28.802041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959779+0000","last_change":"2024-04-25T16:36:26.205090+0000","last_active":"2024-04-25T16:36:27.959779+0000","last_peered":"2024-04-25T16:36:27.959779+0000","last_clean":"2024-04-25T16:36:27.959779+0000","last_became_active":"2024-04-25T16:36:26.204707+0000","last_became_peered":"2024-04-25T16:36:26.204707+0000","last_unstale":"2024-04-25T16:36:27.959779+0000","last_undegraded":"2024-04-25T16:36:27.959779+0000","last_fullsized":"2024-04-25T16:36:27.959779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:09:13.479241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725139+0000","last_change":"2024-04-25T16:36:26.985815+0000","last_active":"2024-04-25T16:36:28.725139+0000","last_peered":"2024-04-25T16:36:28.725139+0000","last_clean":"2024-04-25T16:36:28.725139+0000","last_became_active":"2024-04-25T16:36:26.985472+0000","last_became_peered":"2024-04-25T16:36:26.985472+0000","last_unstale":"2024-04-25T16:36:28.725139+0000","last_undegraded":"2024-04-25T16:36:28.725139+0000","last_fullsized":"2024-04-25T16:36:28.725139+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:33.883655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.958864+0000","last_change":"2024-04-25T16:36:26.207352+0000","last_active":"2024-04-25T16:36:27.958864+0000","last_peered":"2024-04-25T16:36:27.958864+0000","last_clean":"2024-04-25T16:36:27.958864+0000","last_became_active":"2024-04-25T16:36:26.206975+0000","last_became_peered":"2024-04-25T16:36:26.206975+0000","last_unstale":"2024-04-25T16:36:27.958864+0000","last_undegraded":"2024-04-25T16:36:27.958864+0000","last_fullsized":"2024-04-25T16:36:27.958864+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:03.085034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962111+0000","last_change":"2024-04-25T16:36:26.982401+0000","last_active":"2024-04-25T16:36:27.962111+0000","last_peered":"2024-04-25T16:36:27.962111+0000","last_clean":"2024-04-25T16:36:27.962111+0000","last_became_active":"2024-04-25T16:36:26.982070+0000","last_became_peered":"2024-04-25T16:36:26.982070+0000","last_unstale":"2024-04-25T16:36:27.962111+0000","last_undegraded":"2024-04-25T16:36:27.962111+0000","last_fullsized":"2024-04-25T16:36:27.962111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.824276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.958593+0000","last_change":"2024-04-25T16:36:26.214759+0000","last_active":"2024-04-25T16:36:26.958593+0000","last_peered":"2024-04-25T16:36:26.958593+0000","last_clean":"2024-04-25T16:36:26.958593+0000","last_became_active":"2024-04-25T16:36:26.214334+0000","last_became_peered":"2024-04-25T16:36:26.214334+0000","last_unstale":"2024-04-25T16:36:26.958593+0000","last_undegraded":"2024-04-25T16:36:26.958593+0000","last_fullsized":"2024-04-25T16:36:26.958593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:00.537377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.961989+0000","last_change":"2024-04-25T16:36:26.988313+0000","last_active":"2024-04-25T16:36:27.961989+0000","last_peered":"2024-04-25T16:36:27.961989+0000","last_clean":"2024-04-25T16:36:27.961989+0000","last_became_active":"2024-04-25T16:36:26.987979+0000","last_became_peered":"2024-04-25T16:36:26.987979+0000","last_unstale":"2024-04-25T16:36:27.961989+0000","last_undegraded":"2024-04-25T16:36:27.961989+0000","last_fullsized":"2024-04-25T16:36:27.961989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:40.737779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448164+0000","last_change":"2024-04-25T16:36:26.982005+0000","last_active":"2024-04-25T16:36:28.448164+0000","last_peered":"2024-04-25T16:36:28.448164+0000","last_clean":"2024-04-25T16:36:28.448164+0000","last_became_active":"2024-04-25T16:36:26.981619+0000","last_became_peered":"2024-04-25T16:36:26.981619+0000","last_unstale":"2024-04-25T16:36:28.448164+0000","last_undegraded":"2024-04-25T16:36:28.448164+0000","last_fullsized":"2024-04-25T16:36:28.448164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:49.555114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.962128+0000","last_change":"2024-04-25T16:36:26.207188+0000","last_active":"2024-04-25T16:36:26.962128+0000","last_peered":"2024-04-25T16:36:26.962128+0000","last_clean":"2024-04-25T16:36:26.962128+0000","last_became_active":"2024-04-25T16:36:26.206497+0000","last_became_peered":"2024-04-25T16:36:26.206497+0000","last_unstale":"2024-04-25T16:36:26.962128+0000","last_undegraded":"2024-04-25T16:36:26.962128+0000","last_fullsized":"2024-04-25T16:36:26.962128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:45.213263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200712+0000","last_change":"2024-04-25T16:36:26.980524+0000","last_active":"2024-04-25T16:36:29.200712+0000","last_peered":"2024-04-25T16:36:29.200712+0000","last_clean":"2024-04-25T16:36:29.200712+0000","last_became_active":"2024-04-25T16:36:26.980195+0000","last_became_peered":"2024-04-25T16:36:26.980195+0000","last_unstale":"2024-04-25T16:36:29.200712+0000","last_undegraded":"2024-04-25T16:36:29.200712+0000","last_fullsized":"2024-04-25T16:36:29.200712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:25.623403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:31.244218+0000","last_change":"2024-04-25T16:36:26.208423+0000","last_active":"2024-04-25T16:36:31.244218+0000","last_peered":"2024-04-25T16:36:31.244218+0000","last_clean":"2024-04-25T16:36:31.244218+0000","last_became_active":"2024-04-25T16:36:26.208045+0000","last_became_peered":"2024-04-25T16:36:26.208045+0000","last_unstale":"2024-04-25T16:36:31.244218+0000","last_undegraded":"2024-04-25T16:36:31.244218+0000","last_fullsized":"2024-04-25T16:36:31.244218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:47.727762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":88,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.959651+0000","last_change":"2024-04-25T16:35:10.871536+0000","last_active":"2024-04-25T16:36:26.959651+0000","last_peered":"2024-04-25T16:36:26.959651+0000","last_clean":"2024-04-25T16:36:26.959651+0000","last_became_active":"2024-04-25T16:35:10.866041+0000","last_became_peered":"2024-04-25T16:35:10.866041+0000","last_unstale":"2024-04-25T16:36:26.959651+0000","last_undegraded":"2024-04-25T16:36:26.959651+0000","last_fullsized":"2024-04-25T16:36:26.959651+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_clean_scrub_stamp":"2024-04-25T16:31:41.248932+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:45.476632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984646+0000","last_change":"2024-04-25T16:36:26.980701+0000","last_active":"2024-04-25T16:36:26.984646+0000","last_peered":"2024-04-25T16:36:26.984646+0000","last_clean":"2024-04-25T16:36:26.984646+0000","last_became_active":"2024-04-25T16:36:26.980290+0000","last_became_peered":"2024-04-25T16:36:26.980290+0000","last_unstale":"2024-04-25T16:36:26.984646+0000","last_undegraded":"2024-04-25T16:36:26.984646+0000","last_fullsized":"2024-04-25T16:36:26.984646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:07.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.959599+0000","last_change":"2024-04-25T16:36:26.219000+0000","last_active":"2024-04-25T16:36:26.959599+0000","last_peered":"2024-04-25T16:36:26.959599+0000","last_clean":"2024-04-25T16:36:26.959599+0000","last_became_active":"2024-04-25T16:36:26.218599+0000","last_became_peered":"2024-04-25T16:36:26.218599+0000","last_unstale":"2024-04-25T16:36:26.959599+0000","last_undegraded":"2024-04-25T16:36:26.959599+0000","last_fullsized":"2024-04-25T16:36:26.959599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:51.186499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959750+0000","last_change":"2024-04-25T16:36:26.218513+0000","last_active":"2024-04-25T16:36:27.959750+0000","last_peered":"2024-04-25T16:36:27.959750+0000","last_clean":"2024-04-25T16:36:27.959750+0000","last_became_active":"2024-04-25T16:36:26.218157+0000","last_became_peered":"2024-04-25T16:36:26.218157+0000","last_unstale":"2024-04-25T16:36:27.959750+0000","last_undegraded":"2024-04-25T16:36:27.959750+0000","last_fullsized":"2024-04-25T16:36:27.959750+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:40.309818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984593+0000","last_change":"2024-04-25T16:36:26.981916+0000","last_active":"2024-04-25T16:36:26.984593+0000","last_peered":"2024-04-25T16:36:26.984593+0000","last_clean":"2024-04-25T16:36:26.984593+0000","last_became_active":"2024-04-25T16:36:26.978975+0000","last_became_peered":"2024-04-25T16:36:26.978975+0000","last_unstale":"2024-04-25T16:36:26.984593+0000","last_undegraded":"2024-04-25T16:36:26.984593+0000","last_fullsized":"2024-04-25T16:36:26.984593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:10.246787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.235454+0000","last_change":"2024-04-25T16:36:26.221399+0000","last_active":"2024-04-25T16:36:30.235454+0000","last_peered":"2024-04-25T16:36:30.235454+0000","last_clean":"2024-04-25T16:36:30.235454+0000","last_became_active":"2024-04-25T16:36:26.220986+0000","last_became_peered":"2024-04-25T16:36:26.220986+0000","last_unstale":"2024-04-25T16:36:30.235454+0000","last_undegraded":"2024-04-25T16:36:30.235454+0000","last_fullsized":"2024-04-25T16:36:30.235454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:59.793148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200975+0000","last_change":"2024-04-25T16:36:26.974784+0000","last_active":"2024-04-25T16:36:29.200975+0000","last_peered":"2024-04-25T16:36:29.200975+0000","last_clean":"2024-04-25T16:36:29.200975+0000","last_became_active":"2024-04-25T16:36:26.974439+0000","last_became_peered":"2024-04-25T16:36:26.974439+0000","last_unstale":"2024-04-25T16:36:29.200975+0000","last_undegraded":"2024-04-25T16:36:29.200975+0000","last_fullsized":"2024-04-25T16:36:29.200975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:30.552997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.232613+0000","last_change":"2024-04-25T16:36:26.212892+0000","last_active":"2024-04-25T16:36:29.232613+0000","last_peered":"2024-04-25T16:36:29.232613+0000","last_clean":"2024-04-25T16:36:29.232613+0000","last_became_active":"2024-04-25T16:36:26.212443+0000","last_became_peered":"2024-04-25T16:36:26.212443+0000","last_unstale":"2024-04-25T16:36:29.232613+0000","last_undegraded":"2024-04-25T16:36:29.232613+0000","last_fullsized":"2024-04-25T16:36:29.232613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:25.168042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.985670+0000","last_change":"2024-04-25T16:36:26.983776+0000","last_active":"2024-04-25T16:36:26.985670+0000","last_peered":"2024-04-25T16:36:26.985670+0000","last_clean":"2024-04-25T16:36:26.985670+0000","last_became_active":"2024-04-25T16:36:26.983413+0000","last_became_peered":"2024-04-25T16:36:26.983413+0000","last_unstale":"2024-04-25T16:36:26.985670+0000","last_undegraded":"2024-04-25T16:36:26.985670+0000","last_fullsized":"2024-04-25T16:36:26.985670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:11.796016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.231934+0000","last_change":"2024-04-25T16:36:26.212141+0000","last_active":"2024-04-25T16:36:29.231934+0000","last_peered":"2024-04-25T16:36:29.231934+0000","last_clean":"2024-04-25T16:36:29.231934+0000","last_became_active":"2024-04-25T16:36:26.211554+0000","last_became_peered":"2024-04-25T16:36:26.211554+0000","last_unstale":"2024-04-25T16:36:29.231934+0000","last_undegraded":"2024-04-25T16:36:29.231934+0000","last_fullsized":"2024-04-25T16:36:29.231934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:43.970994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959387+0000","last_change":"2024-04-25T16:36:26.978201+0000","last_active":"2024-04-25T16:36:27.959387+0000","last_peered":"2024-04-25T16:36:27.959387+0000","last_clean":"2024-04-25T16:36:27.959387+0000","last_became_active":"2024-04-25T16:36:26.977805+0000","last_became_peered":"2024-04-25T16:36:26.977805+0000","last_unstale":"2024-04-25T16:36:27.959387+0000","last_undegraded":"2024-04-25T16:36:27.959387+0000","last_fullsized":"2024-04-25T16:36:27.959387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:23.852029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.233714+0000","last_change":"2024-04-25T16:36:26.211558+0000","last_active":"2024-04-25T16:36:29.233714+0000","last_peered":"2024-04-25T16:36:29.233714+0000","last_clean":"2024-04-25T16:36:29.233714+0000","last_became_active":"2024-04-25T16:36:26.211191+0000","last_became_peered":"2024-04-25T16:36:26.211191+0000","last_unstale":"2024-04-25T16:36:29.233714+0000","last_undegraded":"2024-04-25T16:36:29.233714+0000","last_fullsized":"2024-04-25T16:36:29.233714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:54.031297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.724913+0000","last_change":"2024-04-25T16:36:26.987439+0000","last_active":"2024-04-25T16:36:28.724913+0000","last_peered":"2024-04-25T16:36:28.724913+0000","last_clean":"2024-04-25T16:36:28.724913+0000","last_became_active":"2024-04-25T16:36:26.986998+0000","last_became_peered":"2024-04-25T16:36:26.986998+0000","last_unstale":"2024-04-25T16:36:28.724913+0000","last_undegraded":"2024-04-25T16:36:28.724913+0000","last_fullsized":"2024-04-25T16:36:28.724913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.057303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.966802+0000","last_change":"2024-04-25T16:36:26.206662+0000","last_active":"2024-04-25T16:36:27.966802+0000","last_peered":"2024-04-25T16:36:27.966802+0000","last_clean":"2024-04-25T16:36:27.966802+0000","last_became_active":"2024-04-25T16:36:26.206237+0000","last_became_peered":"2024-04-25T16:36:26.206237+0000","last_unstale":"2024-04-25T16:36:27.966802+0000","last_undegraded":"2024-04-25T16:36:27.966802+0000","last_fullsized":"2024-04-25T16:36:27.966802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:53.602637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200726+0000","last_change":"2024-04-25T16:36:26.983837+0000","last_active":"2024-04-25T16:36:29.200726+0000","last_peered":"2024-04-25T16:36:29.200726+0000","last_clean":"2024-04-25T16:36:29.200726+0000","last_became_active":"2024-04-25T16:36:26.983513+0000","last_became_peered":"2024-04-25T16:36:26.983513+0000","last_unstale":"2024-04-25T16:36:29.200726+0000","last_undegraded":"2024-04-25T16:36:29.200726+0000","last_fullsized":"2024-04-25T16:36:29.200726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.070866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.227595+0000","last_change":"2024-04-25T16:36:26.216604+0000","last_active":"2024-04-25T16:36:29.227595+0000","last_peered":"2024-04-25T16:36:29.227595+0000","last_clean":"2024-04-25T16:36:29.227595+0000","last_became_active":"2024-04-25T16:36:26.216241+0000","last_became_peered":"2024-04-25T16:36:26.216241+0000","last_unstale":"2024-04-25T16:36:29.227595+0000","last_undegraded":"2024-04-25T16:36:29.227595+0000","last_fullsized":"2024-04-25T16:36:29.227595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:28.157767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.975434+0000","last_change":"2024-04-25T16:36:26.970649+0000","last_active":"2024-04-25T16:36:26.975434+0000","last_peered":"2024-04-25T16:36:26.975434+0000","last_clean":"2024-04-25T16:36:26.975434+0000","last_became_active":"2024-04-25T16:36:26.970266+0000","last_became_peered":"2024-04-25T16:36:26.970266+0000","last_unstale":"2024-04-25T16:36:26.975434+0000","last_undegraded":"2024-04-25T16:36:26.975434+0000","last_fullsized":"2024-04-25T16:36:26.975434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:48.634340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.236705+0000","last_change":"2024-04-25T16:36:26.212419+0000","last_active":"2024-04-25T16:36:30.236705+0000","last_peered":"2024-04-25T16:36:30.236705+0000","last_clean":"2024-04-25T16:36:30.236705+0000","last_became_active":"2024-04-25T16:36:26.211807+0000","last_became_peered":"2024-04-25T16:36:26.211807+0000","last_unstale":"2024-04-25T16:36:30.236705+0000","last_undegraded":"2024-04-25T16:36:30.236705+0000","last_fullsized":"2024-04-25T16:36:30.236705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:02.651503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.977683+0000","last_change":"2024-04-25T16:36:26.970533+0000","last_active":"2024-04-25T16:36:26.977683+0000","last_peered":"2024-04-25T16:36:26.977683+0000","last_clean":"2024-04-25T16:36:26.977683+0000","last_became_active":"2024-04-25T16:36:26.970194+0000","last_became_peered":"2024-04-25T16:36:26.970194+0000","last_unstale":"2024-04-25T16:36:26.977683+0000","last_undegraded":"2024-04-25T16:36:26.977683+0000","last_fullsized":"2024-04-25T16:36:26.977683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:31.980269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448283+0000","last_change":"2024-04-25T16:36:26.210227+0000","last_active":"2024-04-25T16:36:28.448283+0000","last_peered":"2024-04-25T16:36:28.448283+0000","last_clean":"2024-04-25T16:36:28.448283+0000","last_became_active":"2024-04-25T16:36:26.209854+0000","last_became_peered":"2024-04-25T16:36:26.209854+0000","last_unstale":"2024-04-25T16:36:28.448283+0000","last_undegraded":"2024-04-25T16:36:28.448283+0000","last_fullsized":"2024-04-25T16:36:28.448283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:48.542031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.979066+0000","last_change":"2024-04-25T16:36:26.971278+0000","last_active":"2024-04-25T16:36:26.979066+0000","last_peered":"2024-04-25T16:36:26.979066+0000","last_clean":"2024-04-25T16:36:26.979066+0000","last_became_active":"2024-04-25T16:36:26.970959+0000","last_became_peered":"2024-04-25T16:36:26.970959+0000","last_unstale":"2024-04-25T16:36:26.979066+0000","last_undegraded":"2024-04-25T16:36:26.979066+0000","last_fullsized":"2024-04-25T16:36:26.979066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:17.742604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.203329+0000","last_change":"2024-04-25T16:36:26.204435+0000","last_active":"2024-04-25T16:36:27.203329+0000","last_peered":"2024-04-25T16:36:27.203329+0000","last_clean":"2024-04-25T16:36:27.203329+0000","last_became_active":"2024-04-25T16:36:26.203592+0000","last_became_peered":"2024-04-25T16:36:26.203592+0000","last_unstale":"2024-04-25T16:36:27.203329+0000","last_undegraded":"2024-04-25T16:36:27.203329+0000","last_fullsized":"2024-04-25T16:36:27.203329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:18.623888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.975384+0000","last_change":"2024-04-25T16:36:26.971531+0000","last_active":"2024-04-25T16:36:26.975384+0000","last_peered":"2024-04-25T16:36:26.975384+0000","last_clean":"2024-04-25T16:36:26.975384+0000","last_became_active":"2024-04-25T16:36:26.969904+0000","last_became_peered":"2024-04-25T16:36:26.969904+0000","last_unstale":"2024-04-25T16:36:26.975384+0000","last_undegraded":"2024-04-25T16:36:26.975384+0000","last_fullsized":"2024-04-25T16:36:26.975384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:54.804890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725033+0000","last_change":"2024-04-25T16:36:26.981281+0000","last_active":"2024-04-25T16:36:28.725033+0000","last_peered":"2024-04-25T16:36:28.725033+0000","last_clean":"2024-04-25T16:36:28.725033+0000","last_became_active":"2024-04-25T16:36:26.980942+0000","last_became_peered":"2024-04-25T16:36:26.980942+0000","last_unstale":"2024-04-25T16:36:28.725033+0000","last_undegraded":"2024-04-25T16:36:28.725033+0000","last_fullsized":"2024-04-25T16:36:28.725033+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:30.713519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.967035+0000","last_change":"2024-04-25T16:36:26.989589+0000","last_active":"2024-04-25T16:36:27.967035+0000","last_peered":"2024-04-25T16:36:27.967035+0000","last_clean":"2024-04-25T16:36:27.967035+0000","last_became_active":"2024-04-25T16:36:26.989127+0000","last_became_peered":"2024-04-25T16:36:26.989127+0000","last_unstale":"2024-04-25T16:36:27.967035+0000","last_undegraded":"2024-04-25T16:36:27.967035+0000","last_fullsized":"2024-04-25T16:36:27.967035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:53.989995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.966922+0000","last_change":"2024-04-25T16:36:26.983784+0000","last_active":"2024-04-25T16:36:27.966922+0000","last_peered":"2024-04-25T16:36:27.966922+0000","last_clean":"2024-04-25T16:36:27.966922+0000","last_became_active":"2024-04-25T16:36:26.983256+0000","last_became_peered":"2024-04-25T16:36:26.983256+0000","last_unstale":"2024-04-25T16:36:27.966922+0000","last_undegraded":"2024-04-25T16:36:27.966922+0000","last_fullsized":"2024-04-25T16:36:27.966922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:44.469717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.987671+0000","last_change":"2024-04-25T16:36:26.986223+0000","last_active":"2024-04-25T16:36:26.987671+0000","last_peered":"2024-04-25T16:36:26.987671+0000","last_clean":"2024-04-25T16:36:26.987671+0000","last_became_active":"2024-04-25T16:36:26.985511+0000","last_became_peered":"2024-04-25T16:36:26.985511+0000","last_unstale":"2024-04-25T16:36:26.987671+0000","last_undegraded":"2024-04-25T16:36:26.987671+0000","last_fullsized":"2024-04-25T16:36:26.987671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:27.765120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984614+0000","last_change":"2024-04-25T16:36:26.983172+0000","last_active":"2024-04-25T16:36:26.984614+0000","last_peered":"2024-04-25T16:36:26.984614+0000","last_clean":"2024-04-25T16:36:26.984614+0000","last_became_active":"2024-04-25T16:36:26.982793+0000","last_became_peered":"2024-04-25T16:36:26.982793+0000","last_unstale":"2024-04-25T16:36:26.984614+0000","last_undegraded":"2024-04-25T16:36:26.984614+0000","last_fullsized":"2024-04-25T16:36:26.984614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:11.001573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725077+0000","last_change":"2024-04-25T16:36:26.974556+0000","last_active":"2024-04-25T16:36:28.725077+0000","last_peered":"2024-04-25T16:36:28.725077+0000","last_clean":"2024-04-25T16:36:28.725077+0000","last_became_active":"2024-04-25T16:36:26.974179+0000","last_became_peered":"2024-04-25T16:36:26.974179+0000","last_unstale":"2024-04-25T16:36:28.725077+0000","last_undegraded":"2024-04-25T16:36:28.725077+0000","last_fullsized":"2024-04-25T16:36:28.725077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:35.313778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.967437+0000","last_change":"2024-04-25T16:36:26.985614+0000","last_active":"2024-04-25T16:36:27.967437+0000","last_peered":"2024-04-25T16:36:27.967437+0000","last_clean":"2024-04-25T16:36:27.967437+0000","last_became_active":"2024-04-25T16:36:26.985259+0000","last_became_peered":"2024-04-25T16:36:26.985259+0000","last_unstale":"2024-04-25T16:36:27.967437+0000","last_undegraded":"2024-04-25T16:36:27.967437+0000","last_fullsized":"2024-04-25T16:36:27.967437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.850086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200904+0000","last_change":"2024-04-25T16:36:26.981898+0000","last_active":"2024-04-25T16:36:29.200904+0000","last_peered":"2024-04-25T16:36:29.200904+0000","last_clean":"2024-04-25T16:36:29.200904+0000","last_became_active":"2024-04-25T16:36:26.981554+0000","last_became_peered":"2024-04-25T16:36:26.981554+0000","last_unstale":"2024-04-25T16:36:29.200904+0000","last_undegraded":"2024-04-25T16:36:29.200904+0000","last_fullsized":"2024-04-25T16:36:29.200904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:20.836846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.447959+0000","last_change":"2024-04-25T16:36:26.970983+0000","last_active":"2024-04-25T16:36:28.447959+0000","last_peered":"2024-04-25T16:36:28.447959+0000","last_clean":"2024-04-25T16:36:28.447959+0000","last_became_active":"2024-04-25T16:36:26.970644+0000","last_became_peered":"2024-04-25T16:36:26.970644+0000","last_unstale":"2024-04-25T16:36:28.447959+0000","last_undegraded":"2024-04-25T16:36:28.447959+0000","last_fullsized":"2024-04-25T16:36:28.447959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:33.285236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200431+0000","last_change":"2024-04-25T16:36:26.982347+0000","last_active":"2024-04-25T16:36:29.200431+0000","last_peered":"2024-04-25T16:36:29.200431+0000","last_clean":"2024-04-25T16:36:29.200431+0000","last_became_active":"2024-04-25T16:36:26.981970+0000","last_became_peered":"2024-04-25T16:36:26.981970+0000","last_unstale":"2024-04-25T16:36:29.200431+0000","last_undegraded":"2024-04-25T16:36:29.200431+0000","last_fullsized":"2024-04-25T16:36:29.200431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:43.589483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229394+0000","last_change":"2024-04-25T16:36:26.975329+0000","last_active":"2024-04-25T16:36:29.229394+0000","last_peered":"2024-04-25T16:36:29.229394+0000","last_clean":"2024-04-25T16:36:29.229394+0000","last_became_active":"2024-04-25T16:36:26.974791+0000","last_became_peered":"2024-04-25T16:36:26.974791+0000","last_unstale":"2024-04-25T16:36:29.229394+0000","last_undegraded":"2024-04-25T16:36:29.229394+0000","last_fullsized":"2024-04-25T16:36:29.229394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:43.759074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984742+0000","last_change":"2024-04-25T16:36:26.983841+0000","last_active":"2024-04-25T16:36:26.984742+0000","last_peered":"2024-04-25T16:36:26.984742+0000","last_clean":"2024-04-25T16:36:26.984742+0000","last_became_active":"2024-04-25T16:36:26.982595+0000","last_became_peered":"2024-04-25T16:36:26.982595+0000","last_unstale":"2024-04-25T16:36:26.984742+0000","last_undegraded":"2024-04-25T16:36:26.984742+0000","last_fullsized":"2024-04-25T16:36:26.984742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:00.920685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725063+0000","last_change":"2024-04-25T16:36:26.984713+0000","last_active":"2024-04-25T16:36:28.725063+0000","last_peered":"2024-04-25T16:36:28.725063+0000","last_clean":"2024-04-25T16:36:28.725063+0000","last_became_active":"2024-04-25T16:36:26.984330+0000","last_became_peered":"2024-04-25T16:36:26.984330+0000","last_unstale":"2024-04-25T16:36:28.725063+0000","last_undegraded":"2024-04-25T16:36:28.725063+0000","last_fullsized":"2024-04-25T16:36:28.725063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:33.028204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246733,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5321,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":97,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":31066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":802816,"data_stored":384344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":975936,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070489,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":819200,"data_stored":384580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266717,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":384388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":966142,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":786432,"data_stored":375192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":827392,"data_stored":404657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":790528,"data_stored":375282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":386030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":383956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":384406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":975997,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:36:34.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T16:36:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 -- 172.21.15.112:0/4150756695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 msgr2=0x7fe6e0104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/4150756695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e0104010 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802a410 comp rx=0 tx=0).stop 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 -- 172.21.15.112:0/4150756695 shutdown_connections 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/4150756695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e0108900 0x7fe6e010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/4150756695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e0104010 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/4150756695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e01029c0 0x7fe6e01036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 -- 172.21.15.112:0/4150756695 >> 172.21.15.112:0/4150756695 conn(0x7fe6e00fdbd0 msgr2=0x7fe6e0100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 -- 172.21.15.112:0/4150756695 shutdown_connections 2024-04-25T16:36:34.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.210+0000 7fe6e4c51640 1 -- 172.21.15.112:0/4150756695 wait complete. 2024-04-25T16:36:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 Processor -- start 2024-04-25T16:36:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 -- start start 2024-04-25T16:36:34.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e01029c0 0x7fe6e019a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e0108900 0x7fe6e01a1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010e4e0 con 0x7fe6e0103c00 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010e360 con 0x7fe6e01029c0 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.211+0000 7fe6e4c51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e010e660 con 0x7fe6e0108900 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52660/0 (socket says 172.21.15.112:52660) 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 -- 172.21.15.112:0/3316974112 learned_addr learned my addr 172.21.15.112:0/3316974112 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6de59c640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e01029c0 0x7fe6e019a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6ded9d640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e0108900 0x7fe6e01a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 -- 172.21.15.112:0/3316974112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e0108900 msgr2=0x7fe6e01a1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e0108900 0x7fe6e01a1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 -- 172.21.15.112:0/3316974112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e01029c0 msgr2=0x7fe6e019a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e01029c0 0x7fe6e019a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 -- 172.21.15.112:0/3316974112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e01a1da0 con 0x7fe6e0103c00 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6de59c640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e01029c0 0x7fe6e019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d7fff640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e019a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8006010 tx=0x7fe6c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c803f030 con 0x7fe6e0103c00 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6c8031910 con 0x7fe6e0103c00 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e01a2030 con 0x7fe6e0103c00 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8035070 con 0x7fe6e0103c00 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.212+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01a2520 con 0x7fe6e0103c00 2024-04-25T16:36:34.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.213+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e01a2780 con 0x7fe6e0103c00 2024-04-25T16:36:34.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.215+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fe6c8042030 con 0x7fe6e0103c00 2024-04-25T16:36:34.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.216+0000 7fe6d77fe640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe6cc0afb80 0x7fe6cc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:34.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.216+0000 7fe6de59c640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe6cc0afb80 0x7fe6cc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:34.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.216+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fe6c80ff760 con 0x7fe6e0103c00 2024-04-25T16:36:34.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.216+0000 7fe6de59c640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe6cc0afb80 0x7fe6cc0b2090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6e01035c0 tx=0x7fe6d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:34.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.220+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6c80c7d40 con 0x7fe6e0103c00 2024-04-25T16:36:34.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-04-25T16:36:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T16:36:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/917290863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3822053486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:34.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:34 smithi112 ceph-mon[29122]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:36:34.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.396+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6e010a500 con 0x7fe6cc0afb80 2024-04-25T16:36:34.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.403+0000 7fe6d77fe640 1 -- 172.21.15.112:0/3316974112 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258868 (secure 0 0 0) 0x7fe6e010a500 con 0x7fe6cc0afb80 2024-04-25T16:36:34.405 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:34.406 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe6cc0afb80 msgr2=0x7fe6cc0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe6cc0afb80 0x7fe6cc0b2090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe6e01035c0 tx=0x7fe6d0005f90 comp rx=0 tx=0).stop 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 msgr2=0x7fe6e019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e019a7a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8006010 tx=0x7fe6c80042f0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 shutdown_connections 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe6cc0afb80 0x7fe6cc0b2090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e0108900 0x7fe6e01a1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6e0103c00 0x7fe6e019a7a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 --2- 172.21.15.112:0/3316974112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6e01029c0 0x7fe6e019a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 >> 172.21.15.112:0/3316974112 conn(0x7fe6e00fdbd0 msgr2=0x7fe6e0109df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 shutdown_connections 2024-04-25T16:36:34.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:34.406+0000 7fe6e4c51640 1 -- 172.21.15.112:0/3316974112 wait complete. 2024-04-25T16:36:34.416 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-04-25T16:36:33.078947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595689,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":97,"num_read_kb":81,"num_write":220,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":175,"ondisk_log_size":175,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":266,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":346360,"kb_used_data":9640,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124595464,"statfs":{"total":1151940427776,"available":1151585755136,"internally_reserved":0,"allocated":9871360,"data_stored":6380910,"data_compressed":107133,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5321,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":97,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.886048"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725218+0000","last_change":"2024-04-25T16:36:26.988707+0000","last_active":"2024-04-25T16:36:28.725218+0000","last_peered":"2024-04-25T16:36:28.725218+0000","last_clean":"2024-04-25T16:36:28.725218+0000","last_became_active":"2024-04-25T16:36:26.988302+0000","last_became_peered":"2024-04-25T16:36:26.988302+0000","last_unstale":"2024-04-25T16:36:28.725218+0000","last_undegraded":"2024-04-25T16:36:28.725218+0000","last_fullsized":"2024-04-25T16:36:28.725218+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:44.074090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.988824+0000","last_change":"2024-04-25T16:36:26.988147+0000","last_active":"2024-04-25T16:36:26.988824+0000","last_peered":"2024-04-25T16:36:26.988824+0000","last_clean":"2024-04-25T16:36:26.988824+0000","last_became_active":"2024-04-25T16:36:26.987695+0000","last_became_peered":"2024-04-25T16:36:26.987695+0000","last_unstale":"2024-04-25T16:36:26.988824+0000","last_undegraded":"2024-04-25T16:36:26.988824+0000","last_fullsized":"2024-04-25T16:36:26.988824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:16.338402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.988041+0000","last_change":"2024-04-25T16:36:26.986601+0000","last_active":"2024-04-25T16:36:26.988041+0000","last_peered":"2024-04-25T16:36:26.988041+0000","last_clean":"2024-04-25T16:36:26.988041+0000","last_became_active":"2024-04-25T16:36:26.985431+0000","last_became_peered":"2024-04-25T16:36:26.985431+0000","last_unstale":"2024-04-25T16:36:26.988041+0000","last_undegraded":"2024-04-25T16:36:26.988041+0000","last_fullsized":"2024-04-25T16:36:26.988041+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:31:59.454059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229537+0000","last_change":"2024-04-25T16:36:26.985007+0000","last_active":"2024-04-25T16:36:29.229537+0000","last_peered":"2024-04-25T16:36:29.229537+0000","last_clean":"2024-04-25T16:36:29.229537+0000","last_became_active":"2024-04-25T16:36:26.983000+0000","last_became_peered":"2024-04-25T16:36:26.983000+0000","last_unstale":"2024-04-25T16:36:29.229537+0000","last_undegraded":"2024-04-25T16:36:29.229537+0000","last_fullsized":"2024-04-25T16:36:29.229537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:02.324409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959589+0000","last_change":"2024-04-25T16:36:26.970714+0000","last_active":"2024-04-25T16:36:27.959589+0000","last_peered":"2024-04-25T16:36:27.959589+0000","last_clean":"2024-04-25T16:36:27.959589+0000","last_became_active":"2024-04-25T16:36:26.970323+0000","last_became_peered":"2024-04-25T16:36:26.970323+0000","last_unstale":"2024-04-25T16:36:27.959589+0000","last_undegraded":"2024-04-25T16:36:27.959589+0000","last_fullsized":"2024-04-25T16:36:27.959589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:41.113178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.961889+0000","last_change":"2024-04-25T16:36:26.988388+0000","last_active":"2024-04-25T16:36:27.961889+0000","last_peered":"2024-04-25T16:36:27.961889+0000","last_clean":"2024-04-25T16:36:27.961889+0000","last_became_active":"2024-04-25T16:36:26.988090+0000","last_became_peered":"2024-04-25T16:36:26.988090+0000","last_unstale":"2024-04-25T16:36:27.961889+0000","last_undegraded":"2024-04-25T16:36:27.961889+0000","last_fullsized":"2024-04-25T16:36:27.961889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:19.666488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.975018+0000","last_change":"2024-04-25T16:36:26.971244+0000","last_active":"2024-04-25T16:36:26.975018+0000","last_peered":"2024-04-25T16:36:26.975018+0000","last_clean":"2024-04-25T16:36:26.975018+0000","last_became_active":"2024-04-25T16:36:26.970900+0000","last_became_peered":"2024-04-25T16:36:26.970900+0000","last_unstale":"2024-04-25T16:36:26.975018+0000","last_undegraded":"2024-04-25T16:36:26.975018+0000","last_fullsized":"2024-04-25T16:36:26.975018+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:43.262385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200373+0000","last_change":"2024-04-25T16:36:26.982614+0000","last_active":"2024-04-25T16:36:29.200373+0000","last_peered":"2024-04-25T16:36:29.200373+0000","last_clean":"2024-04-25T16:36:29.200373+0000","last_became_active":"2024-04-25T16:36:26.981911+0000","last_became_peered":"2024-04-25T16:36:26.981911+0000","last_unstale":"2024-04-25T16:36:29.200373+0000","last_undegraded":"2024-04-25T16:36:29.200373+0000","last_fullsized":"2024-04-25T16:36:29.200373+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:36.186614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229802+0000","last_change":"2024-04-25T16:36:26.972652+0000","last_active":"2024-04-25T16:36:29.229802+0000","last_peered":"2024-04-25T16:36:29.229802+0000","last_clean":"2024-04-25T16:36:29.229802+0000","last_became_active":"2024-04-25T16:36:26.972332+0000","last_became_peered":"2024-04-25T16:36:26.972332+0000","last_unstale":"2024-04-25T16:36:29.229802+0000","last_undegraded":"2024-04-25T16:36:29.229802+0000","last_fullsized":"2024-04-25T16:36:29.229802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:40.479663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962230+0000","last_change":"2024-04-25T16:36:26.989299+0000","last_active":"2024-04-25T16:36:27.962230+0000","last_peered":"2024-04-25T16:36:27.962230+0000","last_clean":"2024-04-25T16:36:27.962230+0000","last_became_active":"2024-04-25T16:36:26.988826+0000","last_became_peered":"2024-04-25T16:36:26.988826+0000","last_unstale":"2024-04-25T16:36:27.962230+0000","last_undegraded":"2024-04-25T16:36:27.962230+0000","last_fullsized":"2024-04-25T16:36:27.962230+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:40.549866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.977705+0000","last_change":"2024-04-25T16:36:26.971847+0000","last_active":"2024-04-25T16:36:26.977705+0000","last_peered":"2024-04-25T16:36:26.977705+0000","last_clean":"2024-04-25T16:36:26.977705+0000","last_became_active":"2024-04-25T16:36:26.971417+0000","last_became_peered":"2024-04-25T16:36:26.971417+0000","last_unstale":"2024-04-25T16:36:26.977705+0000","last_undegraded":"2024-04-25T16:36:26.977705+0000","last_fullsized":"2024-04-25T16:36:26.977705+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:03.870584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229735+0000","last_change":"2024-04-25T16:36:26.985714+0000","last_active":"2024-04-25T16:36:29.229735+0000","last_peered":"2024-04-25T16:36:29.229735+0000","last_clean":"2024-04-25T16:36:29.229735+0000","last_became_active":"2024-04-25T16:36:26.985402+0000","last_became_peered":"2024-04-25T16:36:26.985402+0000","last_unstale":"2024-04-25T16:36:29.229735+0000","last_undegraded":"2024-04-25T16:36:29.229735+0000","last_fullsized":"2024-04-25T16:36:29.229735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:32.953895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.979214+0000","last_change":"2024-04-25T16:36:26.976917+0000","last_active":"2024-04-25T16:36:26.979214+0000","last_peered":"2024-04-25T16:36:26.979214+0000","last_clean":"2024-04-25T16:36:26.979214+0000","last_became_active":"2024-04-25T16:36:26.976452+0000","last_became_peered":"2024-04-25T16:36:26.976452+0000","last_unstale":"2024-04-25T16:36:26.979214+0000","last_undegraded":"2024-04-25T16:36:26.979214+0000","last_fullsized":"2024-04-25T16:36:26.979214+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:53.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.988461+0000","last_change":"2024-04-25T16:36:26.987433+0000","last_active":"2024-04-25T16:36:26.988461+0000","last_peered":"2024-04-25T16:36:26.988461+0000","last_clean":"2024-04-25T16:36:26.988461+0000","last_became_active":"2024-04-25T16:36:26.987106+0000","last_became_peered":"2024-04-25T16:36:26.987106+0000","last_unstale":"2024-04-25T16:36:26.988461+0000","last_undegraded":"2024-04-25T16:36:26.988461+0000","last_fullsized":"2024-04-25T16:36:26.988461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:57.671061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.986500+0000","last_change":"2024-04-25T16:36:26.984530+0000","last_active":"2024-04-25T16:36:26.986500+0000","last_peered":"2024-04-25T16:36:26.986500+0000","last_clean":"2024-04-25T16:36:26.986500+0000","last_became_active":"2024-04-25T16:36:26.984220+0000","last_became_peered":"2024-04-25T16:36:26.984220+0000","last_unstale":"2024-04-25T16:36:26.986500+0000","last_undegraded":"2024-04-25T16:36:26.986500+0000","last_fullsized":"2024-04-25T16:36:26.986500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:40.810663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.227536+0000","last_change":"2024-04-25T16:36:26.980249+0000","last_active":"2024-04-25T16:36:29.227536+0000","last_peered":"2024-04-25T16:36:29.227536+0000","last_clean":"2024-04-25T16:36:29.227536+0000","last_became_active":"2024-04-25T16:36:26.979861+0000","last_became_peered":"2024-04-25T16:36:26.979861+0000","last_unstale":"2024-04-25T16:36:29.227536+0000","last_undegraded":"2024-04-25T16:36:29.227536+0000","last_fullsized":"2024-04-25T16:36:29.227536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:26.753789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725316+0000","last_change":"2024-04-25T16:36:26.988022+0000","last_active":"2024-04-25T16:36:28.725316+0000","last_peered":"2024-04-25T16:36:28.725316+0000","last_clean":"2024-04-25T16:36:28.725316+0000","last_became_active":"2024-04-25T16:36:26.987609+0000","last_became_peered":"2024-04-25T16:36:26.987609+0000","last_unstale":"2024-04-25T16:36:28.725316+0000","last_undegraded":"2024-04-25T16:36:28.725316+0000","last_fullsized":"2024-04-25T16:36:28.725316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.819517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.958932+0000","last_change":"2024-04-25T16:36:26.978511+0000","last_active":"2024-04-25T16:36:27.958932+0000","last_peered":"2024-04-25T16:36:27.958932+0000","last_clean":"2024-04-25T16:36:27.958932+0000","last_became_active":"2024-04-25T16:36:26.978180+0000","last_became_peered":"2024-04-25T16:36:26.978180+0000","last_unstale":"2024-04-25T16:36:27.958932+0000","last_undegraded":"2024-04-25T16:36:27.958932+0000","last_fullsized":"2024-04-25T16:36:27.958932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:58.931649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.967205+0000","last_change":"2024-04-25T16:36:26.979313+0000","last_active":"2024-04-25T16:36:27.967205+0000","last_peered":"2024-04-25T16:36:27.967205+0000","last_clean":"2024-04-25T16:36:27.967205+0000","last_became_active":"2024-04-25T16:36:26.978958+0000","last_became_peered":"2024-04-25T16:36:26.978958+0000","last_unstale":"2024-04-25T16:36:27.967205+0000","last_undegraded":"2024-04-25T16:36:27.967205+0000","last_fullsized":"2024-04-25T16:36:27.967205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:57.379459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.236580+0000","last_change":"2024-04-25T16:36:26.216356+0000","last_active":"2024-04-25T16:36:30.236580+0000","last_peered":"2024-04-25T16:36:30.236580+0000","last_clean":"2024-04-25T16:36:30.236580+0000","last_became_active":"2024-04-25T16:36:26.215935+0000","last_became_peered":"2024-04-25T16:36:26.215935+0000","last_unstale":"2024-04-25T16:36:30.236580+0000","last_undegraded":"2024-04-25T16:36:30.236580+0000","last_fullsized":"2024-04-25T16:36:30.236580+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:58.186003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200895+0000","last_change":"2024-04-25T16:36:26.981307+0000","last_active":"2024-04-25T16:36:29.200895+0000","last_peered":"2024-04-25T16:36:29.200895+0000","last_clean":"2024-04-25T16:36:29.200895+0000","last_became_active":"2024-04-25T16:36:26.980973+0000","last_became_peered":"2024-04-25T16:36:26.980973+0000","last_unstale":"2024-04-25T16:36:29.200895+0000","last_undegraded":"2024-04-25T16:36:29.200895+0000","last_fullsized":"2024-04-25T16:36:29.200895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:45.469266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.201000+0000","last_change":"2024-04-25T16:36:26.218839+0000","last_active":"2024-04-25T16:36:29.201000+0000","last_peered":"2024-04-25T16:36:29.201000+0000","last_clean":"2024-04-25T16:36:29.201000+0000","last_became_active":"2024-04-25T16:36:26.218487+0000","last_became_peered":"2024-04-25T16:36:26.218487+0000","last_unstale":"2024-04-25T16:36:29.201000+0000","last_undegraded":"2024-04-25T16:36:29.201000+0000","last_fullsized":"2024-04-25T16:36:29.201000+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:05.164084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.981712+0000","last_change":"2024-04-25T16:36:26.977277+0000","last_active":"2024-04-25T16:36:26.981712+0000","last_peered":"2024-04-25T16:36:26.981712+0000","last_clean":"2024-04-25T16:36:26.981712+0000","last_became_active":"2024-04-25T16:36:26.974805+0000","last_became_peered":"2024-04-25T16:36:26.974805+0000","last_unstale":"2024-04-25T16:36:26.981712+0000","last_undegraded":"2024-04-25T16:36:26.981712+0000","last_fullsized":"2024-04-25T16:36:26.981712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:25.219827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959883+0000","last_change":"2024-04-25T16:36:26.215757+0000","last_active":"2024-04-25T16:36:27.959883+0000","last_peered":"2024-04-25T16:36:27.959883+0000","last_clean":"2024-04-25T16:36:27.959883+0000","last_became_active":"2024-04-25T16:36:26.215424+0000","last_became_peered":"2024-04-25T16:36:26.215424+0000","last_unstale":"2024-04-25T16:36:27.959883+0000","last_undegraded":"2024-04-25T16:36:27.959883+0000","last_fullsized":"2024-04-25T16:36:27.959883+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:23.094599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.228561+0000","last_change":"2024-04-25T16:36:26.978552+0000","last_active":"2024-04-25T16:36:29.228561+0000","last_peered":"2024-04-25T16:36:29.228561+0000","last_clean":"2024-04-25T16:36:29.228561+0000","last_became_active":"2024-04-25T16:36:26.978185+0000","last_became_peered":"2024-04-25T16:36:26.978185+0000","last_unstale":"2024-04-25T16:36:29.228561+0000","last_undegraded":"2024-04-25T16:36:29.228561+0000","last_fullsized":"2024-04-25T16:36:29.228561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:19.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959699+0000","last_change":"2024-04-25T16:36:26.207361+0000","last_active":"2024-04-25T16:36:27.959699+0000","last_peered":"2024-04-25T16:36:27.959699+0000","last_clean":"2024-04-25T16:36:27.959699+0000","last_became_active":"2024-04-25T16:36:26.206968+0000","last_became_peered":"2024-04-25T16:36:26.206968+0000","last_unstale":"2024-04-25T16:36:27.959699+0000","last_undegraded":"2024-04-25T16:36:27.959699+0000","last_fullsized":"2024-04-25T16:36:27.959699+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:54.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448323+0000","last_change":"2024-04-25T16:36:26.979723+0000","last_active":"2024-04-25T16:36:28.448323+0000","last_peered":"2024-04-25T16:36:28.448323+0000","last_clean":"2024-04-25T16:36:28.448323+0000","last_became_active":"2024-04-25T16:36:26.979377+0000","last_became_peered":"2024-04-25T16:36:26.979377+0000","last_unstale":"2024-04-25T16:36:28.448323+0000","last_undegraded":"2024-04-25T16:36:28.448323+0000","last_fullsized":"2024-04-25T16:36:28.448323+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:50.312759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:32.248765+0000","last_change":"2024-04-25T16:36:26.217074+0000","last_active":"2024-04-25T16:36:32.248765+0000","last_peered":"2024-04-25T16:36:32.248765+0000","last_clean":"2024-04-25T16:36:32.248765+0000","last_became_active":"2024-04-25T16:36:26.216686+0000","last_became_peered":"2024-04-25T16:36:26.216686+0000","last_unstale":"2024-04-25T16:36:32.248765+0000","last_undegraded":"2024-04-25T16:36:32.248765+0000","last_fullsized":"2024-04-25T16:36:32.248765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:35.780179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959440+0000","last_change":"2024-04-25T16:36:26.972190+0000","last_active":"2024-04-25T16:36:27.959440+0000","last_peered":"2024-04-25T16:36:27.959440+0000","last_clean":"2024-04-25T16:36:27.959440+0000","last_became_active":"2024-04-25T16:36:26.971630+0000","last_became_peered":"2024-04-25T16:36:26.971630+0000","last_unstale":"2024-04-25T16:36:27.959440+0000","last_undegraded":"2024-04-25T16:36:27.959440+0000","last_fullsized":"2024-04-25T16:36:27.959440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:00.009040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.961802+0000","last_change":"2024-04-25T16:36:26.211703+0000","last_active":"2024-04-25T16:36:27.961802+0000","last_peered":"2024-04-25T16:36:27.961802+0000","last_clean":"2024-04-25T16:36:27.961802+0000","last_became_active":"2024-04-25T16:36:26.211322+0000","last_became_peered":"2024-04-25T16:36:26.211322+0000","last_unstale":"2024-04-25T16:36:27.961802+0000","last_undegraded":"2024-04-25T16:36:27.961802+0000","last_fullsized":"2024-04-25T16:36:27.961802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:15.353208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448274+0000","last_change":"2024-04-25T16:36:26.982502+0000","last_active":"2024-04-25T16:36:28.448274+0000","last_peered":"2024-04-25T16:36:28.448274+0000","last_clean":"2024-04-25T16:36:28.448274+0000","last_became_active":"2024-04-25T16:36:26.982112+0000","last_became_peered":"2024-04-25T16:36:26.982112+0000","last_unstale":"2024-04-25T16:36:28.448274+0000","last_undegraded":"2024-04-25T16:36:28.448274+0000","last_fullsized":"2024-04-25T16:36:28.448274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:54.106473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.962201+0000","last_change":"2024-04-25T16:36:26.217225+0000","last_active":"2024-04-25T16:36:26.962201+0000","last_peered":"2024-04-25T16:36:26.962201+0000","last_clean":"2024-04-25T16:36:26.962201+0000","last_became_active":"2024-04-25T16:36:26.216777+0000","last_became_peered":"2024-04-25T16:36:26.216777+0000","last_unstale":"2024-04-25T16:36:26.962201+0000","last_undegraded":"2024-04-25T16:36:26.962201+0000","last_fullsized":"2024-04-25T16:36:26.962201+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:21.433977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200459+0000","last_change":"2024-04-25T16:36:26.970197+0000","last_active":"2024-04-25T16:36:29.200459+0000","last_peered":"2024-04-25T16:36:29.200459+0000","last_clean":"2024-04-25T16:36:29.200459+0000","last_became_active":"2024-04-25T16:36:26.969844+0000","last_became_peered":"2024-04-25T16:36:26.969844+0000","last_unstale":"2024-04-25T16:36:29.200459+0000","last_undegraded":"2024-04-25T16:36:29.200459+0000","last_fullsized":"2024-04-25T16:36:29.200459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:35.397143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.447934+0000","last_change":"2024-04-25T16:36:26.214731+0000","last_active":"2024-04-25T16:36:28.447934+0000","last_peered":"2024-04-25T16:36:28.447934+0000","last_clean":"2024-04-25T16:36:28.447934+0000","last_became_active":"2024-04-25T16:36:26.214343+0000","last_became_peered":"2024-04-25T16:36:26.214343+0000","last_unstale":"2024-04-25T16:36:28.447934+0000","last_undegraded":"2024-04-25T16:36:28.447934+0000","last_fullsized":"2024-04-25T16:36:28.447934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:08:10.156665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.958719+0000","last_change":"2024-04-25T16:36:26.981458+0000","last_active":"2024-04-25T16:36:27.958719+0000","last_peered":"2024-04-25T16:36:27.958719+0000","last_clean":"2024-04-25T16:36:27.958719+0000","last_became_active":"2024-04-25T16:36:26.981116+0000","last_became_peered":"2024-04-25T16:36:26.981116+0000","last_unstale":"2024-04-25T16:36:27.958719+0000","last_undegraded":"2024-04-25T16:36:27.958719+0000","last_fullsized":"2024-04-25T16:36:27.958719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:20.549670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962157+0000","last_change":"2024-04-25T16:36:26.218690+0000","last_active":"2024-04-25T16:36:27.962157+0000","last_peered":"2024-04-25T16:36:27.962157+0000","last_clean":"2024-04-25T16:36:27.962157+0000","last_became_active":"2024-04-25T16:36:26.218287+0000","last_became_peered":"2024-04-25T16:36:26.218287+0000","last_unstale":"2024-04-25T16:36:27.962157+0000","last_undegraded":"2024-04-25T16:36:27.962157+0000","last_fullsized":"2024-04-25T16:36:27.962157+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:42.394413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.230451+0000","last_change":"2024-04-25T16:36:26.980356+0000","last_active":"2024-04-25T16:36:29.230451+0000","last_peered":"2024-04-25T16:36:29.230451+0000","last_clean":"2024-04-25T16:36:29.230451+0000","last_became_active":"2024-04-25T16:36:26.980005+0000","last_became_peered":"2024-04-25T16:36:26.980005+0000","last_unstale":"2024-04-25T16:36:29.230451+0000","last_undegraded":"2024-04-25T16:36:29.230451+0000","last_fullsized":"2024-04-25T16:36:29.230451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:37.263655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.959445+0000","last_change":"2024-04-25T16:36:26.217818+0000","last_active":"2024-04-25T16:36:26.959445+0000","last_peered":"2024-04-25T16:36:26.959445+0000","last_clean":"2024-04-25T16:36:26.959445+0000","last_became_active":"2024-04-25T16:36:26.217317+0000","last_became_peered":"2024-04-25T16:36:26.217317+0000","last_unstale":"2024-04-25T16:36:26.959445+0000","last_undegraded":"2024-04-25T16:36:26.959445+0000","last_fullsized":"2024-04-25T16:36:26.959445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:29.985735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.987652+0000","last_change":"2024-04-25T16:36:26.985901+0000","last_active":"2024-04-25T16:36:26.987652+0000","last_peered":"2024-04-25T16:36:26.987652+0000","last_clean":"2024-04-25T16:36:26.987652+0000","last_became_active":"2024-04-25T16:36:26.983463+0000","last_became_peered":"2024-04-25T16:36:26.983463+0000","last_unstale":"2024-04-25T16:36:26.987652+0000","last_undegraded":"2024-04-25T16:36:26.987652+0000","last_fullsized":"2024-04-25T16:36:26.987652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:40.246613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.231209+0000","last_change":"2024-04-25T16:36:26.208756+0000","last_active":"2024-04-25T16:36:29.231209+0000","last_peered":"2024-04-25T16:36:29.231209+0000","last_clean":"2024-04-25T16:36:29.231209+0000","last_became_active":"2024-04-25T16:36:26.208384+0000","last_became_peered":"2024-04-25T16:36:26.208384+0000","last_unstale":"2024-04-25T16:36:29.231209+0000","last_undegraded":"2024-04-25T16:36:29.231209+0000","last_fullsized":"2024-04-25T16:36:29.231209+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.849402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.228445+0000","last_change":"2024-04-25T16:36:26.981611+0000","last_active":"2024-04-25T16:36:29.228445+0000","last_peered":"2024-04-25T16:36:29.228445+0000","last_clean":"2024-04-25T16:36:29.228445+0000","last_became_active":"2024-04-25T16:36:26.981311+0000","last_became_peered":"2024-04-25T16:36:26.981311+0000","last_unstale":"2024-04-25T16:36:29.228445+0000","last_undegraded":"2024-04-25T16:36:29.228445+0000","last_fullsized":"2024-04-25T16:36:29.228445+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:05.360035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"74'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962222+0000","last_change":"2024-04-25T16:36:26.214215+0000","last_active":"2024-04-25T16:36:27.962222+0000","last_peered":"2024-04-25T16:36:27.962222+0000","last_clean":"2024-04-25T16:36:27.962222+0000","last_became_active":"2024-04-25T16:36:26.213848+0000","last_became_peered":"2024-04-25T16:36:26.213848+0000","last_unstale":"2024-04-25T16:36:27.962222+0000","last_undegraded":"2024-04-25T16:36:27.962222+0000","last_fullsized":"2024-04-25T16:36:27.962222+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:32.589646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959796+0000","last_change":"2024-04-25T16:36:26.976087+0000","last_active":"2024-04-25T16:36:27.959796+0000","last_peered":"2024-04-25T16:36:27.959796+0000","last_clean":"2024-04-25T16:36:27.959796+0000","last_became_active":"2024-04-25T16:36:26.975726+0000","last_became_peered":"2024-04-25T16:36:26.975726+0000","last_unstale":"2024-04-25T16:36:27.959796+0000","last_undegraded":"2024-04-25T16:36:27.959796+0000","last_fullsized":"2024-04-25T16:36:27.959796+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:02.668930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.232489+0000","last_change":"2024-04-25T16:36:26.208815+0000","last_active":"2024-04-25T16:36:30.232489+0000","last_peered":"2024-04-25T16:36:30.232489+0000","last_clean":"2024-04-25T16:36:30.232489+0000","last_became_active":"2024-04-25T16:36:26.208485+0000","last_became_peered":"2024-04-25T16:36:26.208485+0000","last_unstale":"2024-04-25T16:36:30.232489+0000","last_undegraded":"2024-04-25T16:36:30.232489+0000","last_fullsized":"2024-04-25T16:36:30.232489+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:11.984900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.976692+0000","last_change":"2024-04-25T16:36:26.975567+0000","last_active":"2024-04-25T16:36:26.976692+0000","last_peered":"2024-04-25T16:36:26.976692+0000","last_clean":"2024-04-25T16:36:26.976692+0000","last_became_active":"2024-04-25T16:36:26.975212+0000","last_became_peered":"2024-04-25T16:36:26.975212+0000","last_unstale":"2024-04-25T16:36:26.976692+0000","last_undegraded":"2024-04-25T16:36:26.976692+0000","last_fullsized":"2024-04-25T16:36:26.976692+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:15:09.010964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.233052+0000","last_change":"2024-04-25T16:36:26.214765+0000","last_active":"2024-04-25T16:36:29.233052+0000","last_peered":"2024-04-25T16:36:29.233052+0000","last_clean":"2024-04-25T16:36:29.233052+0000","last_became_active":"2024-04-25T16:36:26.214378+0000","last_became_peered":"2024-04-25T16:36:26.214378+0000","last_unstale":"2024-04-25T16:36:29.233052+0000","last_undegraded":"2024-04-25T16:36:29.233052+0000","last_fullsized":"2024-04-25T16:36:29.233052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:00.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.981932+0000","last_change":"2024-04-25T16:36:26.978074+0000","last_active":"2024-04-25T16:36:26.981932+0000","last_peered":"2024-04-25T16:36:26.981932+0000","last_clean":"2024-04-25T16:36:26.981932+0000","last_became_active":"2024-04-25T16:36:26.977706+0000","last_became_peered":"2024-04-25T16:36:26.977706+0000","last_unstale":"2024-04-25T16:36:26.981932+0000","last_undegraded":"2024-04-25T16:36:26.981932+0000","last_fullsized":"2024-04-25T16:36:26.981932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:44.591511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:31.248003+0000","last_change":"2024-04-25T16:36:26.211526+0000","last_active":"2024-04-25T16:36:31.248003+0000","last_peered":"2024-04-25T16:36:31.248003+0000","last_clean":"2024-04-25T16:36:31.248003+0000","last_became_active":"2024-04-25T16:36:26.211134+0000","last_became_peered":"2024-04-25T16:36:26.211134+0000","last_unstale":"2024-04-25T16:36:31.248003+0000","last_undegraded":"2024-04-25T16:36:31.248003+0000","last_fullsized":"2024-04-25T16:36:31.248003+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:25:39.743843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.981989+0000","last_change":"2024-04-25T16:36:26.979766+0000","last_active":"2024-04-25T16:36:26.981989+0000","last_peered":"2024-04-25T16:36:26.981989+0000","last_clean":"2024-04-25T16:36:26.981989+0000","last_became_active":"2024-04-25T16:36:26.979460+0000","last_became_peered":"2024-04-25T16:36:26.979460+0000","last_unstale":"2024-04-25T16:36:26.981989+0000","last_undegraded":"2024-04-25T16:36:26.981989+0000","last_fullsized":"2024-04-25T16:36:26.981989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:21.081975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'10","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.228207+0000","last_change":"2024-04-25T16:36:26.212112+0000","last_active":"2024-04-25T16:36:29.228207+0000","last_peered":"2024-04-25T16:36:29.228207+0000","last_clean":"2024-04-25T16:36:29.228207+0000","last_became_active":"2024-04-25T16:36:26.211746+0000","last_became_peered":"2024-04-25T16:36:26.211746+0000","last_unstale":"2024-04-25T16:36:29.228207+0000","last_undegraded":"2024-04-25T16:36:29.228207+0000","last_fullsized":"2024-04-25T16:36:29.228207+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:24.682525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448052+0000","last_change":"2024-04-25T16:36:26.980275+0000","last_active":"2024-04-25T16:36:28.448052+0000","last_peered":"2024-04-25T16:36:28.448052+0000","last_clean":"2024-04-25T16:36:28.448052+0000","last_became_active":"2024-04-25T16:36:26.979912+0000","last_became_peered":"2024-04-25T16:36:26.979912+0000","last_unstale":"2024-04-25T16:36:28.448052+0000","last_undegraded":"2024-04-25T16:36:28.448052+0000","last_fullsized":"2024-04-25T16:36:28.448052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:28.802041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"74'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959779+0000","last_change":"2024-04-25T16:36:26.205090+0000","last_active":"2024-04-25T16:36:27.959779+0000","last_peered":"2024-04-25T16:36:27.959779+0000","last_clean":"2024-04-25T16:36:27.959779+0000","last_became_active":"2024-04-25T16:36:26.204707+0000","last_became_peered":"2024-04-25T16:36:26.204707+0000","last_unstale":"2024-04-25T16:36:27.959779+0000","last_undegraded":"2024-04-25T16:36:27.959779+0000","last_fullsized":"2024-04-25T16:36:27.959779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:09:13.479241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725139+0000","last_change":"2024-04-25T16:36:26.985815+0000","last_active":"2024-04-25T16:36:28.725139+0000","last_peered":"2024-04-25T16:36:28.725139+0000","last_clean":"2024-04-25T16:36:28.725139+0000","last_became_active":"2024-04-25T16:36:26.985472+0000","last_became_peered":"2024-04-25T16:36:26.985472+0000","last_unstale":"2024-04-25T16:36:28.725139+0000","last_undegraded":"2024-04-25T16:36:28.725139+0000","last_fullsized":"2024-04-25T16:36:28.725139+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:33.883655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.958864+0000","last_change":"2024-04-25T16:36:26.207352+0000","last_active":"2024-04-25T16:36:27.958864+0000","last_peered":"2024-04-25T16:36:27.958864+0000","last_clean":"2024-04-25T16:36:27.958864+0000","last_became_active":"2024-04-25T16:36:26.206975+0000","last_became_peered":"2024-04-25T16:36:26.206975+0000","last_unstale":"2024-04-25T16:36:27.958864+0000","last_undegraded":"2024-04-25T16:36:27.958864+0000","last_fullsized":"2024-04-25T16:36:27.958864+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:03.085034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.962111+0000","last_change":"2024-04-25T16:36:26.982401+0000","last_active":"2024-04-25T16:36:27.962111+0000","last_peered":"2024-04-25T16:36:27.962111+0000","last_clean":"2024-04-25T16:36:27.962111+0000","last_became_active":"2024-04-25T16:36:26.982070+0000","last_became_peered":"2024-04-25T16:36:26.982070+0000","last_unstale":"2024-04-25T16:36:27.962111+0000","last_undegraded":"2024-04-25T16:36:27.962111+0000","last_fullsized":"2024-04-25T16:36:27.962111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.824276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.958593+0000","last_change":"2024-04-25T16:36:26.214759+0000","last_active":"2024-04-25T16:36:26.958593+0000","last_peered":"2024-04-25T16:36:26.958593+0000","last_clean":"2024-04-25T16:36:26.958593+0000","last_became_active":"2024-04-25T16:36:26.214334+0000","last_became_peered":"2024-04-25T16:36:26.214334+0000","last_unstale":"2024-04-25T16:36:26.958593+0000","last_undegraded":"2024-04-25T16:36:26.958593+0000","last_fullsized":"2024-04-25T16:36:26.958593+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:00.537377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.961989+0000","last_change":"2024-04-25T16:36:26.988313+0000","last_active":"2024-04-25T16:36:27.961989+0000","last_peered":"2024-04-25T16:36:27.961989+0000","last_clean":"2024-04-25T16:36:27.961989+0000","last_became_active":"2024-04-25T16:36:26.987979+0000","last_became_peered":"2024-04-25T16:36:26.987979+0000","last_unstale":"2024-04-25T16:36:27.961989+0000","last_undegraded":"2024-04-25T16:36:27.961989+0000","last_fullsized":"2024-04-25T16:36:27.961989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:40.737779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448164+0000","last_change":"2024-04-25T16:36:26.982005+0000","last_active":"2024-04-25T16:36:28.448164+0000","last_peered":"2024-04-25T16:36:28.448164+0000","last_clean":"2024-04-25T16:36:28.448164+0000","last_became_active":"2024-04-25T16:36:26.981619+0000","last_became_peered":"2024-04-25T16:36:26.981619+0000","last_unstale":"2024-04-25T16:36:28.448164+0000","last_undegraded":"2024-04-25T16:36:28.448164+0000","last_fullsized":"2024-04-25T16:36:28.448164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:49.555114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.962128+0000","last_change":"2024-04-25T16:36:26.207188+0000","last_active":"2024-04-25T16:36:26.962128+0000","last_peered":"2024-04-25T16:36:26.962128+0000","last_clean":"2024-04-25T16:36:26.962128+0000","last_became_active":"2024-04-25T16:36:26.206497+0000","last_became_peered":"2024-04-25T16:36:26.206497+0000","last_unstale":"2024-04-25T16:36:26.962128+0000","last_undegraded":"2024-04-25T16:36:26.962128+0000","last_fullsized":"2024-04-25T16:36:26.962128+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:45.213263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200712+0000","last_change":"2024-04-25T16:36:26.980524+0000","last_active":"2024-04-25T16:36:29.200712+0000","last_peered":"2024-04-25T16:36:29.200712+0000","last_clean":"2024-04-25T16:36:29.200712+0000","last_became_active":"2024-04-25T16:36:26.980195+0000","last_became_peered":"2024-04-25T16:36:26.980195+0000","last_unstale":"2024-04-25T16:36:29.200712+0000","last_undegraded":"2024-04-25T16:36:29.200712+0000","last_fullsized":"2024-04-25T16:36:29.200712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:25.623403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:31.244218+0000","last_change":"2024-04-25T16:36:26.208423+0000","last_active":"2024-04-25T16:36:31.244218+0000","last_peered":"2024-04-25T16:36:31.244218+0000","last_clean":"2024-04-25T16:36:31.244218+0000","last_became_active":"2024-04-25T16:36:26.208045+0000","last_became_peered":"2024-04-25T16:36:26.208045+0000","last_unstale":"2024-04-25T16:36:31.244218+0000","last_undegraded":"2024-04-25T16:36:31.244218+0000","last_fullsized":"2024-04-25T16:36:31.244218+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:47.727762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":88,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.959651+0000","last_change":"2024-04-25T16:35:10.871536+0000","last_active":"2024-04-25T16:36:26.959651+0000","last_peered":"2024-04-25T16:36:26.959651+0000","last_clean":"2024-04-25T16:36:26.959651+0000","last_became_active":"2024-04-25T16:35:10.866041+0000","last_became_peered":"2024-04-25T16:35:10.866041+0000","last_unstale":"2024-04-25T16:36:26.959651+0000","last_undegraded":"2024-04-25T16:36:26.959651+0000","last_fullsized":"2024-04-25T16:36:26.959651+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_clean_scrub_stamp":"2024-04-25T16:31:41.248932+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:45.476632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984646+0000","last_change":"2024-04-25T16:36:26.980701+0000","last_active":"2024-04-25T16:36:26.984646+0000","last_peered":"2024-04-25T16:36:26.984646+0000","last_clean":"2024-04-25T16:36:26.984646+0000","last_became_active":"2024-04-25T16:36:26.980290+0000","last_became_peered":"2024-04-25T16:36:26.980290+0000","last_unstale":"2024-04-25T16:36:26.984646+0000","last_undegraded":"2024-04-25T16:36:26.984646+0000","last_fullsized":"2024-04-25T16:36:26.984646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:07.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.959599+0000","last_change":"2024-04-25T16:36:26.219000+0000","last_active":"2024-04-25T16:36:26.959599+0000","last_peered":"2024-04-25T16:36:26.959599+0000","last_clean":"2024-04-25T16:36:26.959599+0000","last_became_active":"2024-04-25T16:36:26.218599+0000","last_became_peered":"2024-04-25T16:36:26.218599+0000","last_unstale":"2024-04-25T16:36:26.959599+0000","last_undegraded":"2024-04-25T16:36:26.959599+0000","last_fullsized":"2024-04-25T16:36:26.959599+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:51.186499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959750+0000","last_change":"2024-04-25T16:36:26.218513+0000","last_active":"2024-04-25T16:36:27.959750+0000","last_peered":"2024-04-25T16:36:27.959750+0000","last_clean":"2024-04-25T16:36:27.959750+0000","last_became_active":"2024-04-25T16:36:26.218157+0000","last_became_peered":"2024-04-25T16:36:26.218157+0000","last_unstale":"2024-04-25T16:36:27.959750+0000","last_undegraded":"2024-04-25T16:36:27.959750+0000","last_fullsized":"2024-04-25T16:36:27.959750+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:40.309818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984593+0000","last_change":"2024-04-25T16:36:26.981916+0000","last_active":"2024-04-25T16:36:26.984593+0000","last_peered":"2024-04-25T16:36:26.984593+0000","last_clean":"2024-04-25T16:36:26.984593+0000","last_became_active":"2024-04-25T16:36:26.978975+0000","last_became_peered":"2024-04-25T16:36:26.978975+0000","last_unstale":"2024-04-25T16:36:26.984593+0000","last_undegraded":"2024-04-25T16:36:26.984593+0000","last_fullsized":"2024-04-25T16:36:26.984593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:10.246787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.235454+0000","last_change":"2024-04-25T16:36:26.221399+0000","last_active":"2024-04-25T16:36:30.235454+0000","last_peered":"2024-04-25T16:36:30.235454+0000","last_clean":"2024-04-25T16:36:30.235454+0000","last_became_active":"2024-04-25T16:36:26.220986+0000","last_became_peered":"2024-04-25T16:36:26.220986+0000","last_unstale":"2024-04-25T16:36:30.235454+0000","last_undegraded":"2024-04-25T16:36:30.235454+0000","last_fullsized":"2024-04-25T16:36:30.235454+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:59.793148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200975+0000","last_change":"2024-04-25T16:36:26.974784+0000","last_active":"2024-04-25T16:36:29.200975+0000","last_peered":"2024-04-25T16:36:29.200975+0000","last_clean":"2024-04-25T16:36:29.200975+0000","last_became_active":"2024-04-25T16:36:26.974439+0000","last_became_peered":"2024-04-25T16:36:26.974439+0000","last_unstale":"2024-04-25T16:36:29.200975+0000","last_undegraded":"2024-04-25T16:36:29.200975+0000","last_fullsized":"2024-04-25T16:36:29.200975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:30.552997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.232613+0000","last_change":"2024-04-25T16:36:26.212892+0000","last_active":"2024-04-25T16:36:29.232613+0000","last_peered":"2024-04-25T16:36:29.232613+0000","last_clean":"2024-04-25T16:36:29.232613+0000","last_became_active":"2024-04-25T16:36:26.212443+0000","last_became_peered":"2024-04-25T16:36:26.212443+0000","last_unstale":"2024-04-25T16:36:29.232613+0000","last_undegraded":"2024-04-25T16:36:29.232613+0000","last_fullsized":"2024-04-25T16:36:29.232613+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:25.168042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.985670+0000","last_change":"2024-04-25T16:36:26.983776+0000","last_active":"2024-04-25T16:36:26.985670+0000","last_peered":"2024-04-25T16:36:26.985670+0000","last_clean":"2024-04-25T16:36:26.985670+0000","last_became_active":"2024-04-25T16:36:26.983413+0000","last_became_peered":"2024-04-25T16:36:26.983413+0000","last_unstale":"2024-04-25T16:36:26.985670+0000","last_undegraded":"2024-04-25T16:36:26.985670+0000","last_fullsized":"2024-04-25T16:36:26.985670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:11.796016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.231934+0000","last_change":"2024-04-25T16:36:26.212141+0000","last_active":"2024-04-25T16:36:29.231934+0000","last_peered":"2024-04-25T16:36:29.231934+0000","last_clean":"2024-04-25T16:36:29.231934+0000","last_became_active":"2024-04-25T16:36:26.211554+0000","last_became_peered":"2024-04-25T16:36:26.211554+0000","last_unstale":"2024-04-25T16:36:29.231934+0000","last_undegraded":"2024-04-25T16:36:29.231934+0000","last_fullsized":"2024-04-25T16:36:29.231934+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:43.970994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.959387+0000","last_change":"2024-04-25T16:36:26.978201+0000","last_active":"2024-04-25T16:36:27.959387+0000","last_peered":"2024-04-25T16:36:27.959387+0000","last_clean":"2024-04-25T16:36:27.959387+0000","last_became_active":"2024-04-25T16:36:26.977805+0000","last_became_peered":"2024-04-25T16:36:26.977805+0000","last_unstale":"2024-04-25T16:36:27.959387+0000","last_undegraded":"2024-04-25T16:36:27.959387+0000","last_fullsized":"2024-04-25T16:36:27.959387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:23.852029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.233714+0000","last_change":"2024-04-25T16:36:26.211558+0000","last_active":"2024-04-25T16:36:29.233714+0000","last_peered":"2024-04-25T16:36:29.233714+0000","last_clean":"2024-04-25T16:36:29.233714+0000","last_became_active":"2024-04-25T16:36:26.211191+0000","last_became_peered":"2024-04-25T16:36:26.211191+0000","last_unstale":"2024-04-25T16:36:29.233714+0000","last_undegraded":"2024-04-25T16:36:29.233714+0000","last_fullsized":"2024-04-25T16:36:29.233714+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:54.031297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.724913+0000","last_change":"2024-04-25T16:36:26.987439+0000","last_active":"2024-04-25T16:36:28.724913+0000","last_peered":"2024-04-25T16:36:28.724913+0000","last_clean":"2024-04-25T16:36:28.724913+0000","last_became_active":"2024-04-25T16:36:26.986998+0000","last_became_peered":"2024-04-25T16:36:26.986998+0000","last_unstale":"2024-04-25T16:36:28.724913+0000","last_undegraded":"2024-04-25T16:36:28.724913+0000","last_fullsized":"2024-04-25T16:36:28.724913+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.057303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":16,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.966802+0000","last_change":"2024-04-25T16:36:26.206662+0000","last_active":"2024-04-25T16:36:27.966802+0000","last_peered":"2024-04-25T16:36:27.966802+0000","last_clean":"2024-04-25T16:36:27.966802+0000","last_became_active":"2024-04-25T16:36:26.206237+0000","last_became_peered":"2024-04-25T16:36:26.206237+0000","last_unstale":"2024-04-25T16:36:27.966802+0000","last_undegraded":"2024-04-25T16:36:27.966802+0000","last_fullsized":"2024-04-25T16:36:27.966802+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:53.602637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200726+0000","last_change":"2024-04-25T16:36:26.983837+0000","last_active":"2024-04-25T16:36:29.200726+0000","last_peered":"2024-04-25T16:36:29.200726+0000","last_clean":"2024-04-25T16:36:29.200726+0000","last_became_active":"2024-04-25T16:36:26.983513+0000","last_became_peered":"2024-04-25T16:36:26.983513+0000","last_unstale":"2024-04-25T16:36:29.200726+0000","last_undegraded":"2024-04-25T16:36:29.200726+0000","last_fullsized":"2024-04-25T16:36:29.200726+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.070866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.227595+0000","last_change":"2024-04-25T16:36:26.216604+0000","last_active":"2024-04-25T16:36:29.227595+0000","last_peered":"2024-04-25T16:36:29.227595+0000","last_clean":"2024-04-25T16:36:29.227595+0000","last_became_active":"2024-04-25T16:36:26.216241+0000","last_became_peered":"2024-04-25T16:36:26.216241+0000","last_unstale":"2024-04-25T16:36:29.227595+0000","last_undegraded":"2024-04-25T16:36:29.227595+0000","last_fullsized":"2024-04-25T16:36:29.227595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:28.157767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.975434+0000","last_change":"2024-04-25T16:36:26.970649+0000","last_active":"2024-04-25T16:36:26.975434+0000","last_peered":"2024-04-25T16:36:26.975434+0000","last_clean":"2024-04-25T16:36:26.975434+0000","last_became_active":"2024-04-25T16:36:26.970266+0000","last_became_peered":"2024-04-25T16:36:26.970266+0000","last_unstale":"2024-04-25T16:36:26.975434+0000","last_undegraded":"2024-04-25T16:36:26.975434+0000","last_fullsized":"2024-04-25T16:36:26.975434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:48.634340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":18,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:30.236705+0000","last_change":"2024-04-25T16:36:26.212419+0000","last_active":"2024-04-25T16:36:30.236705+0000","last_peered":"2024-04-25T16:36:30.236705+0000","last_clean":"2024-04-25T16:36:30.236705+0000","last_became_active":"2024-04-25T16:36:26.211807+0000","last_became_peered":"2024-04-25T16:36:26.211807+0000","last_unstale":"2024-04-25T16:36:30.236705+0000","last_undegraded":"2024-04-25T16:36:30.236705+0000","last_fullsized":"2024-04-25T16:36:30.236705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:02.651503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.977683+0000","last_change":"2024-04-25T16:36:26.970533+0000","last_active":"2024-04-25T16:36:26.977683+0000","last_peered":"2024-04-25T16:36:26.977683+0000","last_clean":"2024-04-25T16:36:26.977683+0000","last_became_active":"2024-04-25T16:36:26.970194+0000","last_became_peered":"2024-04-25T16:36:26.970194+0000","last_unstale":"2024-04-25T16:36:26.977683+0000","last_undegraded":"2024-04-25T16:36:26.977683+0000","last_fullsized":"2024-04-25T16:36:26.977683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:31.980269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.448283+0000","last_change":"2024-04-25T16:36:26.210227+0000","last_active":"2024-04-25T16:36:28.448283+0000","last_peered":"2024-04-25T16:36:28.448283+0000","last_clean":"2024-04-25T16:36:28.448283+0000","last_became_active":"2024-04-25T16:36:26.209854+0000","last_became_peered":"2024-04-25T16:36:26.209854+0000","last_unstale":"2024-04-25T16:36:28.448283+0000","last_undegraded":"2024-04-25T16:36:28.448283+0000","last_fullsized":"2024-04-25T16:36:28.448283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:48.542031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.979066+0000","last_change":"2024-04-25T16:36:26.971278+0000","last_active":"2024-04-25T16:36:26.979066+0000","last_peered":"2024-04-25T16:36:26.979066+0000","last_clean":"2024-04-25T16:36:26.979066+0000","last_became_active":"2024-04-25T16:36:26.970959+0000","last_became_peered":"2024-04-25T16:36:26.970959+0000","last_unstale":"2024-04-25T16:36:26.979066+0000","last_undegraded":"2024-04-25T16:36:26.979066+0000","last_fullsized":"2024-04-25T16:36:26.979066+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:17.742604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.203329+0000","last_change":"2024-04-25T16:36:26.204435+0000","last_active":"2024-04-25T16:36:27.203329+0000","last_peered":"2024-04-25T16:36:27.203329+0000","last_clean":"2024-04-25T16:36:27.203329+0000","last_became_active":"2024-04-25T16:36:26.203592+0000","last_became_peered":"2024-04-25T16:36:26.203592+0000","last_unstale":"2024-04-25T16:36:27.203329+0000","last_undegraded":"2024-04-25T16:36:27.203329+0000","last_fullsized":"2024-04-25T16:36:27.203329+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:18.623888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.975384+0000","last_change":"2024-04-25T16:36:26.971531+0000","last_active":"2024-04-25T16:36:26.975384+0000","last_peered":"2024-04-25T16:36:26.975384+0000","last_clean":"2024-04-25T16:36:26.975384+0000","last_became_active":"2024-04-25T16:36:26.969904+0000","last_became_peered":"2024-04-25T16:36:26.969904+0000","last_unstale":"2024-04-25T16:36:26.975384+0000","last_undegraded":"2024-04-25T16:36:26.975384+0000","last_fullsized":"2024-04-25T16:36:26.975384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:54.804890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725033+0000","last_change":"2024-04-25T16:36:26.981281+0000","last_active":"2024-04-25T16:36:28.725033+0000","last_peered":"2024-04-25T16:36:28.725033+0000","last_clean":"2024-04-25T16:36:28.725033+0000","last_became_active":"2024-04-25T16:36:26.980942+0000","last_became_peered":"2024-04-25T16:36:26.980942+0000","last_unstale":"2024-04-25T16:36:28.725033+0000","last_undegraded":"2024-04-25T16:36:28.725033+0000","last_fullsized":"2024-04-25T16:36:28.725033+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:30.713519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.967035+0000","last_change":"2024-04-25T16:36:26.989589+0000","last_active":"2024-04-25T16:36:27.967035+0000","last_peered":"2024-04-25T16:36:27.967035+0000","last_clean":"2024-04-25T16:36:27.967035+0000","last_became_active":"2024-04-25T16:36:26.989127+0000","last_became_peered":"2024-04-25T16:36:26.989127+0000","last_unstale":"2024-04-25T16:36:27.967035+0000","last_undegraded":"2024-04-25T16:36:27.967035+0000","last_fullsized":"2024-04-25T16:36:27.967035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:53.989995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.966922+0000","last_change":"2024-04-25T16:36:26.983784+0000","last_active":"2024-04-25T16:36:27.966922+0000","last_peered":"2024-04-25T16:36:27.966922+0000","last_clean":"2024-04-25T16:36:27.966922+0000","last_became_active":"2024-04-25T16:36:26.983256+0000","last_became_peered":"2024-04-25T16:36:26.983256+0000","last_unstale":"2024-04-25T16:36:27.966922+0000","last_undegraded":"2024-04-25T16:36:27.966922+0000","last_fullsized":"2024-04-25T16:36:27.966922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:44.469717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.987671+0000","last_change":"2024-04-25T16:36:26.986223+0000","last_active":"2024-04-25T16:36:26.987671+0000","last_peered":"2024-04-25T16:36:26.987671+0000","last_clean":"2024-04-25T16:36:26.987671+0000","last_became_active":"2024-04-25T16:36:26.985511+0000","last_became_peered":"2024-04-25T16:36:26.985511+0000","last_unstale":"2024-04-25T16:36:26.987671+0000","last_undegraded":"2024-04-25T16:36:26.987671+0000","last_fullsized":"2024-04-25T16:36:26.987671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:27.765120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984614+0000","last_change":"2024-04-25T16:36:26.983172+0000","last_active":"2024-04-25T16:36:26.984614+0000","last_peered":"2024-04-25T16:36:26.984614+0000","last_clean":"2024-04-25T16:36:26.984614+0000","last_became_active":"2024-04-25T16:36:26.982793+0000","last_became_peered":"2024-04-25T16:36:26.982793+0000","last_unstale":"2024-04-25T16:36:26.984614+0000","last_undegraded":"2024-04-25T16:36:26.984614+0000","last_fullsized":"2024-04-25T16:36:26.984614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:11.001573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725077+0000","last_change":"2024-04-25T16:36:26.974556+0000","last_active":"2024-04-25T16:36:28.725077+0000","last_peered":"2024-04-25T16:36:28.725077+0000","last_clean":"2024-04-25T16:36:28.725077+0000","last_became_active":"2024-04-25T16:36:26.974179+0000","last_became_peered":"2024-04-25T16:36:26.974179+0000","last_unstale":"2024-04-25T16:36:28.725077+0000","last_undegraded":"2024-04-25T16:36:28.725077+0000","last_fullsized":"2024-04-25T16:36:28.725077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:35.313778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:27.967437+0000","last_change":"2024-04-25T16:36:26.985614+0000","last_active":"2024-04-25T16:36:27.967437+0000","last_peered":"2024-04-25T16:36:27.967437+0000","last_clean":"2024-04-25T16:36:27.967437+0000","last_became_active":"2024-04-25T16:36:26.985259+0000","last_became_peered":"2024-04-25T16:36:26.985259+0000","last_unstale":"2024-04-25T16:36:27.967437+0000","last_undegraded":"2024-04-25T16:36:27.967437+0000","last_fullsized":"2024-04-25T16:36:27.967437+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.850086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200904+0000","last_change":"2024-04-25T16:36:26.981898+0000","last_active":"2024-04-25T16:36:29.200904+0000","last_peered":"2024-04-25T16:36:29.200904+0000","last_clean":"2024-04-25T16:36:29.200904+0000","last_became_active":"2024-04-25T16:36:26.981554+0000","last_became_peered":"2024-04-25T16:36:26.981554+0000","last_unstale":"2024-04-25T16:36:29.200904+0000","last_undegraded":"2024-04-25T16:36:29.200904+0000","last_fullsized":"2024-04-25T16:36:29.200904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:20.836846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.447959+0000","last_change":"2024-04-25T16:36:26.970983+0000","last_active":"2024-04-25T16:36:28.447959+0000","last_peered":"2024-04-25T16:36:28.447959+0000","last_clean":"2024-04-25T16:36:28.447959+0000","last_became_active":"2024-04-25T16:36:26.970644+0000","last_became_peered":"2024-04-25T16:36:26.970644+0000","last_unstale":"2024-04-25T16:36:28.447959+0000","last_undegraded":"2024-04-25T16:36:28.447959+0000","last_fullsized":"2024-04-25T16:36:28.447959+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:33.285236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.200431+0000","last_change":"2024-04-25T16:36:26.982347+0000","last_active":"2024-04-25T16:36:29.200431+0000","last_peered":"2024-04-25T16:36:29.200431+0000","last_clean":"2024-04-25T16:36:29.200431+0000","last_became_active":"2024-04-25T16:36:26.981970+0000","last_became_peered":"2024-04-25T16:36:26.981970+0000","last_unstale":"2024-04-25T16:36:29.200431+0000","last_undegraded":"2024-04-25T16:36:29.200431+0000","last_fullsized":"2024-04-25T16:36:29.200431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:43.589483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:29.229394+0000","last_change":"2024-04-25T16:36:26.975329+0000","last_active":"2024-04-25T16:36:29.229394+0000","last_peered":"2024-04-25T16:36:29.229394+0000","last_clean":"2024-04-25T16:36:29.229394+0000","last_became_active":"2024-04-25T16:36:26.974791+0000","last_became_peered":"2024-04-25T16:36:26.974791+0000","last_unstale":"2024-04-25T16:36:29.229394+0000","last_undegraded":"2024-04-25T16:36:29.229394+0000","last_fullsized":"2024-04-25T16:36:29.229394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:43.759074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-04-25T16:36:26.984742+0000","last_change":"2024-04-25T16:36:26.983841+0000","last_active":"2024-04-25T16:36:26.984742+0000","last_peered":"2024-04-25T16:36:26.984742+0000","last_clean":"2024-04-25T16:36:26.984742+0000","last_became_active":"2024-04-25T16:36:26.982595+0000","last_became_peered":"2024-04-25T16:36:26.982595+0000","last_unstale":"2024-04-25T16:36:26.984742+0000","last_undegraded":"2024-04-25T16:36:26.984742+0000","last_fullsized":"2024-04-25T16:36:26.984742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:00.920685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T16:36:28.725063+0000","last_change":"2024-04-25T16:36:26.984713+0000","last_active":"2024-04-25T16:36:28.725063+0000","last_peered":"2024-04-25T16:36:28.725063+0000","last_clean":"2024-04-25T16:36:28.725063+0000","last_became_active":"2024-04-25T16:36:26.984330+0000","last_became_peered":"2024-04-25T16:36:26.984330+0000","last_unstale":"2024-04-25T16:36:28.725063+0000","last_undegraded":"2024-04-25T16:36:28.725063+0000","last_fullsized":"2024-04-25T16:36:28.725063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:33.028204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246733,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5321,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":97,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":31066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710734,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":802816,"data_stored":384344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906963,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":975936,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070489,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":819200,"data_stored":384580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266717,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":384388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":966142,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626472,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31952,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713200,"statfs":{"total":95995035648,"available":95962316800,"internally_reserved":0,"allocated":786432,"data_stored":375192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":827392,"data_stored":404657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":790528,"data_stored":375282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":386030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":383956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986227,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":811008,"data_stored":384406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":975997,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:36:34.420 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T16:36:34.421 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T16:36:34.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T16:36:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-04-25T16:36:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T16:36:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/917290863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3822053486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:34.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:34 smithi116 ceph-mon[32712]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:36:34.691 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:34.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: pgmap v284: 97 pgs: 97 active+clean; 582 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4.4 KiB/s wr, 16 op/s 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/917290863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3822053486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:34 smithi133 ceph-mon[32691]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:36:35.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:35 smithi112 ceph-mon[29122]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:36:35.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:35 smithi112 ceph-mon[29122]: pgmap v285: 97 pgs: 97 active+clean; 618 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.9 KiB/s wr, 20 op/s 2024-04-25T16:36:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:35 smithi133 ceph-mon[32691]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:36:35.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:35 smithi133 ceph-mon[32691]: pgmap v285: 97 pgs: 97 active+clean; 618 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.9 KiB/s wr, 20 op/s 2024-04-25T16:36:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:35 smithi116 ceph-mon[32712]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:36:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:35 smithi116 ceph-mon[32712]: pgmap v285: 97 pgs: 97 active+clean; 618 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 8.9 KiB/s wr, 20 op/s 2024-04-25T16:36:36.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 -- 172.21.15.112:0/507693240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 msgr2=0x7fefa0109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 --2- 172.21.15.112:0/507693240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa0109a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fef88009990 tx=0x7fef88009ca0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 -- 172.21.15.112:0/507693240 shutdown_connections 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 --2- 172.21.15.112:0/507693240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 0x7fefa010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 --2- 172.21.15.112:0/507693240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 0x7fefa0107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 --2- 172.21.15.112:0/507693240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa0109a80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 -- 172.21.15.112:0/507693240 >> 172.21.15.112:0/507693240 conn(0x7fefa00fe740 msgr2=0x7fefa0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.404+0000 7fefa53ce700 1 -- 172.21.15.112:0/507693240 shutdown_connections 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.405+0000 7fefa53ce700 1 -- 172.21.15.112:0/507693240 wait complete. 2024-04-25T16:36:36.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.405+0000 7fefa53ce700 1 Processor -- start 2024-04-25T16:36:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.405+0000 7fefa53ce700 1 -- start start 2024-04-25T16:36:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fefa53ce700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 0x7fefa019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fefa53ce700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fefa53ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 0x7fefa01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:36.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fefa53ce700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fefa010fc70 con 0x7fefa0107660 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fefa53ce700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fefa010faf0 con 0x7fefa0109fc0 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fefa53ce700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fefa010fdf0 con 0x7fefa0104d00 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60508/0 (socket says 172.21.15.112:60508) 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 -- 172.21.15.112:0/1401895373 learned_addr learned my addr 172.21.15.112:0/1401895373 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef9effd700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 0x7fefa019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef9f7fe700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 0x7fefa01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 -- 172.21.15.112:0/1401895373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 msgr2=0x7fefa019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 0x7fefa019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 -- 172.21.15.112:0/1401895373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 msgr2=0x7fefa01a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 0x7fefa01a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 -- 172.21.15.112:0/1401895373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa01a22f0 con 0x7fefa0107660 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef9f7fe700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 0x7fefa01a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef9effd700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 0x7fefa019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef967fc700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa019acd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef88006010 tx=0x7fef880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:36.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.406+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef88027070 con 0x7fefa0107660 2024-04-25T16:36:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.407+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa01a2580 con 0x7fefa0107660 2024-04-25T16:36:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.407+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef88004140 con 0x7fefa0107660 2024-04-25T16:36:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.407+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8800f460 con 0x7fefa0107660 2024-04-25T16:36:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.407+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa01a2a30 con 0x7fefa0107660 2024-04-25T16:36:36.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.408+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa0051190 con 0x7fefa0107660 2024-04-25T16:36:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.409+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fef8802a070 con 0x7fefa0107660 2024-04-25T16:36:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.410+0000 7fef9cff9700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fef8c0afc10 0x7fef8c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.410+0000 7fef9effd700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fef8c0afc10 0x7fef8c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.410+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fef880dff80 con 0x7fefa0107660 2024-04-25T16:36:36.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.410+0000 7fef9effd700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fef8c0afc10 0x7fef8c0b2120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef90005fd0 tx=0x7fef90005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:36.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.414+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef88013070 con 0x7fefa0107660 2024-04-25T16:36:36.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.693+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fefa002e510 con 0x7fefa0107660 2024-04-25T16:36:36.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.696+0000 7fef9cff9700 1 -- 172.21.15.112:0/1401895373 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fef880a8bc0 con 0x7fefa0107660 2024-04-25T16:36:36.694 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-25T16:36:36.695 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQCUhipmSsVqKRAADLlNrRMOD05hdXSSlShPyQ== 2024-04-25T16:36:36.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fef8c0afc10 msgr2=0x7fef8c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fef8c0afc10 0x7fef8c0b2120 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fef90005fd0 tx=0x7fef90005ee0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 msgr2=0x7fefa019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa019acd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fef88006010 tx=0x7fef880047b0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 shutdown_connections 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fef8c0afc10 0x7fef8c0b2120 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fefa0109fc0 0x7fefa01a1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefa0104d00 0x7fefa019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 --2- 172.21.15.112:0/1401895373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fefa0107660 0x7fefa019acd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 >> 172.21.15.112:0/1401895373 conn(0x7fefa00fe740 msgr2=0x7fefa01057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:36.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 shutdown_connections 2024-04-25T16:36:36.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:36.698+0000 7fefa53ce700 1 -- 172.21.15.112:0/1401895373 wait complete. 2024-04-25T16:36:37.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:36 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1401895373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T16:36:37.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:36 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1401895373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T16:36:37.020 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:36 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1401895373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T16:36:37.020 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:36 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1401895373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T16:36:37.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:36 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1401895373' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T16:36:37.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:36 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1401895373' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T16:36:37.347 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:36:37.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T16:36:37.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T16:36:37.382 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T16:36:37.382 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T16:36:37.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T16:36:37.658 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:38.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:37 smithi116 ceph-mon[32712]: pgmap v286: 97 pgs: 97 active+clean; 618 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-25T16:36:38.017 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:37 smithi112 ceph-mon[29122]: pgmap v286: 97 pgs: 97 active+clean; 618 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-25T16:36:38.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:37 smithi133 ceph-mon[32691]: pgmap v286: 97 pgs: 97 active+clean; 618 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 8.0 KiB/s wr, 17 op/s 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 -- 172.21.15.112:0/3069148731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 msgr2=0x7fcb8c1091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 --2- 172.21.15.112:0/3069148731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c1091e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0099c0 tx=0x7fcb7c009cd0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 -- 172.21.15.112:0/3069148731 shutdown_connections 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 --2- 172.21.15.112:0/3069148731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8c109720 0x7fcb8c111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 --2- 172.21.15.112:0/3069148731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb8c1044a0 0x7fcb8c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 --2- 172.21.15.112:0/3069148731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c1091e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 -- 172.21.15.112:0/3069148731 >> 172.21.15.112:0/3069148731 conn(0x7fcb8c0ffd70 msgr2=0x7fcb8c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 -- 172.21.15.112:0/3069148731 shutdown_connections 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.471+0000 7fcb9366d700 1 -- 172.21.15.112:0/3069148731 wait complete. 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.472+0000 7fcb9366d700 1 Processor -- start 2024-04-25T16:36:39.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.472+0000 7fcb9366d700 1 -- start start 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.472+0000 7fcb9366d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8c1044a0 0x7fcb8c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.472+0000 7fcb9366d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb9366d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb8c109720 0x7fcb8c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb9366d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c114330 con 0x7fcb8c108dd0 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb9366d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1141b0 con 0x7fcb8c1044a0 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb9366d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c1144b0 con 0x7fcb8c109720 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb91409700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8c1044a0 0x7fcb8c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60534/0 (socket says 172.21.15.112:60534) 2024-04-25T16:36:39.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 -- 172.21.15.112:0/1041350047 learned_addr learned my addr 172.21.15.112:0/1041350047 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb91c0a700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb8c109720 0x7fcb8c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 -- 172.21.15.112:0/1041350047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb8c109720 msgr2=0x7fcb8c19f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb8c109720 0x7fcb8c19f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 -- 172.21.15.112:0/1041350047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8c1044a0 msgr2=0x7fcb8c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8c1044a0 0x7fcb8c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 -- 172.21.15.112:0/1041350047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c19fc30 con 0x7fcb8c108dd0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb90c08700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c19b300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c027070 con 0x7fcb8c108dd0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb7c003f20 con 0x7fcb8c108dd0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.473+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c19fec0 con 0x7fcb8c108dd0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.474+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c022c60 con 0x7fcb8c108dd0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.474+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c1a7840 con 0x7fcb8c108dd0 2024-04-25T16:36:39.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.474+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c105950 con 0x7fcb8c108dd0 2024-04-25T16:36:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.476+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fcb7c02a070 con 0x7fcb8c108dd0 2024-04-25T16:36:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.477+0000 7fcb827fc700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fcb780afc90 0x7fcb780b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.477+0000 7fcb91409700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fcb780afc90 0x7fcb780b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:39.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.477+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fcb7c022e00 con 0x7fcb8c108dd0 2024-04-25T16:36:39.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.477+0000 7fcb91409700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fcb780afc90 0x7fcb780b21a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007be0 tx=0x7fcb880073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:39.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.481+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb7c031080 con 0x7fcb8c108dd0 2024-04-25T16:36:39.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.737+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcb8c051190 con 0x7fcb8c108dd0 2024-04-25T16:36:39.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.743+0000 7fcb827fc700 1 -- 172.21.15.112:0/1041350047 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+284486 (secure 0 0 0) 0x7fcb7c013070 con 0x7fcb8c108dd0 2024-04-25T16:36:39.746 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.745+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fcb780afc90 msgr2=0x7fcb780b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.745+0000 7fcb9366d700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fcb780afc90 0x7fcb780b21a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007be0 tx=0x7fcb880073d0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.745+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 msgr2=0x7fcb8c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.745+0000 7fcb9366d700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c19b300 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c0047b0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 shutdown_connections 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb8c109720 0x7fcb8c19f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fcb780afc90 0x7fcb780b21a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcb8c1044a0 0x7fcb8c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 --2- 172.21.15.112:0/1041350047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcb8c108dd0 0x7fcb8c19b300 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 >> 172.21.15.112:0/1041350047 conn(0x7fcb8c0ffd70 msgr2=0x7fcb8c10a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:39.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 shutdown_connections 2024-04-25T16:36:39.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:39.746+0000 7fcb9366d700 1 -- 172.21.15.112:0/1041350047 wait complete. 2024-04-25T16:36:40.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:40 smithi133 ceph-mon[32691]: pgmap v287: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2024-04-25T16:36:40.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:40 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1041350047' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T16:36:40.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:40 smithi116 ceph-mon[32712]: pgmap v287: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2024-04-25T16:36:40.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:40 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1041350047' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T16:36:40.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:40 smithi112 ceph-mon[29122]: pgmap v287: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.6 KiB/s wr, 17 op/s 2024-04-25T16:36:40.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:40 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1041350047' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T16:36:40.561 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3947590362},{"type":"v1","addr":"172.21.15.112:6801","nonce":3947590362}]},"active_addr":"172.21.15.112:6801/3947590362","active_change":"2024-04-25T16:29:14.986401+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.112:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":386718339}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":4216056022}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":795264311}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2445069646}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":524163965}]}]} 2024-04-25T16:36:40.568 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T16:36:40.568 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T16:36:40.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T16:36:40.806 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:42.336 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:42 smithi112 ceph-mon[29122]: pgmap v288: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-04-25T16:36:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:42 smithi133 ceph-mon[32691]: pgmap v288: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-04-25T16:36:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:42 smithi116 ceph-mon[32712]: pgmap v288: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.8 KiB/s wr, 16 op/s 2024-04-25T16:36:42.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.574+0000 7fc29da29700 1 -- 172.21.15.112:0/2735770700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2981096e0 msgr2=0x7fc298111040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:42.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.574+0000 7fc29da29700 1 --2- 172.21.15.112:0/2735770700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2981096e0 0x7fc298111040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc2880099c0 tx=0x7fc288009cd0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 -- 172.21.15.112:0/2735770700 shutdown_connections 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 --2- 172.21.15.112:0/2735770700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2981096e0 0x7fc298111040 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 --2- 172.21.15.112:0/2735770700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 0x7fc298104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 --2- 172.21.15.112:0/2735770700 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc298108d90 0x7fc2981091a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 -- 172.21.15.112:0/2735770700 >> 172.21.15.112:0/2735770700 conn(0x7fc2980ffd40 msgr2=0x7fc298102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 -- 172.21.15.112:0/2735770700 shutdown_connections 2024-04-25T16:36:42.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 -- 172.21.15.112:0/2735770700 wait complete. 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 Processor -- start 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.575+0000 7fc29da29700 1 -- start start 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29da29700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 0x7fc29819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29da29700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 0x7fc29819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29da29700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2981096e0 0x7fc29819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29da29700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc298114400 con 0x7fc298108d90 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29da29700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc298114280 con 0x7fc298104460 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29da29700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc298114580 con 0x7fc2981096e0 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 0x7fc29819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 0x7fc29819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60540/0 (socket says 172.21.15.112:60540) 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 -- 172.21.15.112:0/3584533249 learned_addr learned my addr 172.21.15.112:0/3584533249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc296ffd700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 0x7fc29819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:42.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2977fe700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2981096e0 0x7fc29819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 -- 172.21.15.112:0/3584533249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2981096e0 msgr2=0x7fc29819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2981096e0 0x7fc29819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 -- 172.21.15.112:0/3584533249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 msgr2=0x7fc29819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 0x7fc29819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 -- 172.21.15.112:0/3584533249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29819fbe0 con 0x7fc298108d90 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc296ffd700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 0x7fc29819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc2967fc700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 0x7fc29819b2b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00b500 tx=0x7fc28c00b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.576+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c015020 con 0x7fc298108d90 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.577+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29819fed0 con 0x7fc298108d90 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.577+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc28c011ce0 con 0x7fc298108d90 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.577+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c010d40 con 0x7fc298108d90 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.577+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2981a77f0 con 0x7fc298108d90 2024-04-25T16:36:42.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.578+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc298051190 con 0x7fc298108d90 2024-04-25T16:36:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.579+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fc28c018020 con 0x7fc298108d90 2024-04-25T16:36:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.580+0000 7fc29ca27700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc2800afc10 0x7fc2800b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.580+0000 7fc296ffd700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc2800afc10 0x7fc2800b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.580+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc28c0de8b0 con 0x7fc298108d90 2024-04-25T16:36:42.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.580+0000 7fc296ffd700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc2800afc10 0x7fc2800b2120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc290007eb0 tx=0x7fc2900073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:42.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.583+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc28c01f020 con 0x7fc298108d90 2024-04-25T16:36:42.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.787+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc29819c070 con 0x7fc298108d90 2024-04-25T16:36:42.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.788+0000 7fc29ca27700 1 -- 172.21.15.112:0/3584533249 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22299 (secure 0 0 0) 0x7fc28c0a75f0 con 0x7fc298108d90 2024-04-25T16:36:42.788 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:42.788 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":75,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:36:27.955247+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc2800afc10 msgr2=0x7fc2800b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc2800afc10 0x7fc2800b2120 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc290007eb0 tx=0x7fc2900073c0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 msgr2=0x7fc29819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 0x7fc29819b2b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00b500 tx=0x7fc28c00b8c0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 shutdown_connections 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2981096e0 0x7fc29819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc2800afc10 0x7fc2800b2120 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc298104460 0x7fc29819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 --2- 172.21.15.112:0/3584533249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc298108d90 0x7fc29819b2b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:42.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 >> 172.21.15.112:0/3584533249 conn(0x7fc2980ffd40 msgr2=0x7fc298108500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:42.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 shutdown_connections 2024-04-25T16:36:42.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:42.791+0000 7fc29da29700 1 -- 172.21.15.112:0/3584533249 wait complete. 2024-04-25T16:36:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:43 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3584533249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:43 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3584533249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:43.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:43 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3584533249' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:43.673 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T16:36:43.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T16:36:43.908 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:44.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:44 smithi112 ceph-mon[29122]: pgmap v289: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-04-25T16:36:44.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:44 smithi133 ceph-mon[32691]: pgmap v289: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-04-25T16:36:44.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:44 smithi116 ceph-mon[32712]: pgmap v289: 97 pgs: 97 active+clean; 623 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.675+0000 7f599485b700 1 -- 172.21.15.112:0/2494246916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5990107c90 msgr2=0x7f59901080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.675+0000 7f599485b700 1 --2- 172.21.15.112:0/2494246916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5990107c90 0x7f59901080a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f59840099c0 tx=0x7f5984009cd0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 -- 172.21.15.112:0/2494246916 shutdown_connections 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 --2- 172.21.15.112:0/2494246916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59901085e0 0x7f599010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 --2- 172.21.15.112:0/2494246916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5990103360 0x7f5990103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 --2- 172.21.15.112:0/2494246916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5990107c90 0x7f59901080a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 -- 172.21.15.112:0/2494246916 >> 172.21.15.112:0/2494246916 conn(0x7f59900fec90 msgr2=0x7f59901010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 -- 172.21.15.112:0/2494246916 shutdown_connections 2024-04-25T16:36:45.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.676+0000 7f599485b700 1 -- 172.21.15.112:0/2494246916 wait complete. 2024-04-25T16:36:45.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 Processor -- start 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 -- start start 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5990103360 0x7f599019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5990107c90 0x7f599019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 0x7f599019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5990114360 con 0x7f59901085e0 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59901141e0 con 0x7f5990103360 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f599485b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59901144e0 con 0x7f5990107c90 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f598ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 0x7f599019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f598ed9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 0x7f599019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48636/0 (socket says 172.21.15.112:48636) 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f598ed9d700 1 -- 172.21.15.112:0/615231386 learned_addr learned my addr 172.21.15.112:0/615231386 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f598dd9b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5990107c90 0x7f599019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:45.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.677+0000 7f598e59c700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5990103360 0x7f599019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598ed9d700 1 -- 172.21.15.112:0/615231386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5990107c90 msgr2=0x7f599019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598ed9d700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5990107c90 0x7f599019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598ed9d700 1 -- 172.21.15.112:0/615231386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5990103360 msgr2=0x7f599019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598ed9d700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5990103360 0x7f599019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598ed9d700 1 -- 172.21.15.112:0/615231386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599019fc80 con 0x7f59901085e0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598e59c700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5990103360 0x7f599019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f598ed9d700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 0x7f599019f5e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5988007ee0 tx=0x7f598800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598800dbd0 con 0x7f59901085e0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599019ff70 con 0x7f59901085e0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598800dd70 con 0x7f59901085e0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59901a7890 con 0x7f59901085e0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.678+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988005990 con 0x7f59901085e0 2024-04-25T16:36:45.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.679+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5990104810 con 0x7f59901085e0 2024-04-25T16:36:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.681+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f598801a020 con 0x7f59901085e0 2024-04-25T16:36:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.681+0000 7f597f7fe700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f59780afc10 0x7f59780b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.681+0000 7f598e59c700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f59780afc10 0x7f59780b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:45.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.681+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f59880dea70 con 0x7f59901085e0 2024-04-25T16:36:45.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.682+0000 7f598e59c700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f59780afc10 0x7f59780b2120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59800099d0 tx=0x7f5980008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:45.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.686+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5988010070 con 0x7f59901085e0 2024-04-25T16:36:45.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.882+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f599019c110 con 0x7f59901085e0 2024-04-25T16:36:45.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.883+0000 7f597f7fe700 1 -- 172.21.15.112:0/615231386 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22299 (secure 0 0 0) 0x7f59880a76a0 con 0x7f59901085e0 2024-04-25T16:36:45.882 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:36:45.883 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":75,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:36:27.955247+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:36:45.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f59780afc10 msgr2=0x7f59780b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:45.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f59780afc10 0x7f59780b2120 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f59800099d0 tx=0x7f5980008040 comp rx=0 tx=0).stop 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 msgr2=0x7f599019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 0x7f599019f5e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5988007ee0 tx=0x7f598800d3b0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 shutdown_connections 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59901085e0 0x7f599019f5e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f59780afc10 0x7f59780b2120 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5990103360 0x7f599019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 --2- 172.21.15.112:0/615231386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5990107c90 0x7f599019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 >> 172.21.15.112:0/615231386 conn(0x7f59900fec90 msgr2=0x7f5990107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 shutdown_connections 2024-04-25T16:36:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:45.886+0000 7f599485b700 1 -- 172.21.15.112:0/615231386 wait complete. 2024-04-25T16:36:46.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:46 smithi133 ceph-mon[32691]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-25T16:36:46.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:46 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/615231386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:46.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:46 smithi112 ceph-mon[29122]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-25T16:36:46.508 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:46 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/615231386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:46 smithi116 ceph-mon[32712]: pgmap v290: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 6.1 KiB/s wr, 11 op/s 2024-04-25T16:36:46.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:46 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/615231386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:36:46.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T16:36:46.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T16:36:46.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T16:36:46.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T16:36:46.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T16:36:46.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T16:36:46.538 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T16:36:46.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T16:36:46.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T16:36:46.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T16:36:46.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T16:36:46.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T16:36:47.315 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.077 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.078 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.079 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.082 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.083 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.085 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.085 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.087 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.089 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:48.316 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:48 smithi112 ceph-mon[29122]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-04-25T16:36:48.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:48 smithi133 ceph-mon[32691]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-04-25T16:36:48.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:48 smithi116 ceph-mon[32712]: pgmap v291: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.6 KiB/s wr, 7 op/s 2024-04-25T16:36:48.656 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:49.357 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:36:50.409 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:50 smithi112 ceph-mon[29122]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T16:36:50.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:50 smithi133 ceph-mon[32691]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T16:36:50.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:50 smithi116 ceph-mon[32712]: pgmap v292: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T16:36:52.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:52 smithi133 ceph-mon[32691]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T16:36:52.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:52 smithi112 ceph-mon[29122]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T16:36:52.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:52 smithi116 ceph-mon[32712]: pgmap v293: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T16:36:54.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:54 smithi133 ceph-mon[32691]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T16:36:54.467 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:54 smithi112 ceph-mon[29122]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T16:36:54.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:54 smithi116 ceph-mon[32712]: pgmap v294: 97 pgs: 97 active+clean; 635 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 -- 172.21.15.112:0/1740313524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff790103360 msgr2=0x7ff790103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 --2- 172.21.15.112:0/1740313524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff790103360 0x7ff790103770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff7800099c0 tx=0x7ff780009cd0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 -- 172.21.15.112:0/1740313524 shutdown_connections 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 --2- 172.21.15.112:0/1740313524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7901085e0 0x7ff79010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 --2- 172.21.15.112:0/1740313524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff790103360 0x7ff790103770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 --2- 172.21.15.112:0/1740313524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff790107c90 0x7ff7901080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 -- 172.21.15.112:0/1740313524 >> 172.21.15.112:0/1740313524 conn(0x7ff7900fec90 msgr2=0x7ff7901010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 -- 172.21.15.112:0/1740313524 shutdown_connections 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.742+0000 7ff796764700 1 -- 172.21.15.112:0/1740313524 wait complete. 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.743+0000 7ff796764700 1 Processor -- start 2024-04-25T16:36:56.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.743+0000 7ff796764700 1 -- start start 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff796764700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff790103360 0x7ff7901a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff796764700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff790107c90 0x7ff7901a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff796764700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7901085e0 0x7ff79019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff796764700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff790113210 con 0x7ff7901085e0 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff796764700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff790113090 con 0x7ff790103360 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff796764700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff790113390 con 0x7ff790107c90 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7901085e0 0x7ff79019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7901085e0 0x7ff79019aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48656/0 (socket says 172.21.15.112:48656) 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 -- 172.21.15.112:0/3770074469 learned_addr learned my addr 172.21.15.112:0/3770074469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff78f7fe700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff790107c90 0x7ff7901a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff78ffff700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff790103360 0x7ff7901a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 -- 172.21.15.112:0/3770074469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff790107c90 msgr2=0x7ff7901a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff790107c90 0x7ff7901a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 -- 172.21.15.112:0/3770074469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff790103360 msgr2=0x7ff7901a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff790103360 0x7ff7901a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff794d01700 1 -- 172.21.15.112:0/3770074469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79019b150 con 0x7ff7901085e0 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.744+0000 7ff78f7fe700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff790107c90 0x7ff7901a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.745+0000 7ff794d01700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7901085e0 0x7ff79019aaa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff78800ef30 tx=0x7ff78800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.745+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788004bb0 con 0x7ff7901085e0 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.745+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79019b440 con 0x7ff7901085e0 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.745+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff788004d50 con 0x7ff7901085e0 2024-04-25T16:36:56.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.745+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788004280 con 0x7ff7901085e0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.745+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7900ff0a0 con 0x7ff7901085e0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.746+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff790063e20 con 0x7ff7901085e0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.748+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7ff788003a00 con 0x7ff7901085e0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78d7fa700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff7780afc10 0x7ff7780b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78ffff700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff7780afc10 0x7ff7780b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff7880e29a0 con 0x7ff7901085e0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78d7fa700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] conn(0x7ff7780b9440 0x7ff7780bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 --> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff788025e50 con 0x7ff7780b9440 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff7880e2d50 con 0x7ff7901085e0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78f7fe700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] conn(0x7ff7780b9440 0x7ff7780bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.749+0000 7ff78ffff700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff7780afc10 0x7ff7780b2120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff780000c00 tx=0x7ff78001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.750+0000 7ff78f7fe700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] conn(0x7ff7780b9440 0x7ff7780bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.750+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== osd.3 v2:172.21.15.112:6810/1393578642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff788025e50 con 0x7ff7780b9440 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.767+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 --> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff790064220 con 0x7ff7780b9440 2024-04-25T16:36:56.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff78d7fa700 1 -- 172.21.15.112:0/3770074469 <== osd.3 v2:172.21.15.112:6810/1393578642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff790064220 con 0x7ff7780b9440 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] conn(0x7ff7780b9440 msgr2=0x7ff7780bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff796764700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] conn(0x7ff7780b9440 0x7ff7780bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff7780afc10 msgr2=0x7ff7780b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff796764700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff7780afc10 0x7ff7780b2120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff780000c00 tx=0x7ff78001a040 comp rx=0 tx=0).stop 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7901085e0 msgr2=0x7ff79019aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff796764700 1 --2- 172.21.15.112:0/3770074469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff7901085e0 0x7ff79019aaa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff78800ef30 tx=0x7ff78800c3a0 comp rx=0 tx=0).stop 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.768+0000 7ff794d01700 1 -- 172.21.15.112:0/3770074469 reap_dead start 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.769+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 shutdown_connections 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.769+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 >> 172.21.15.112:0/3770074469 conn(0x7ff7900fec90 msgr2=0x7ff79006b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.769+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 shutdown_connections 2024-04-25T16:36:56.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:50.769+0000 7ff796764700 1 -- 172.21.15.112:0/3770074469 wait complete. 2024-04-25T16:36:56.448 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:56 smithi112 ceph-mon[29122]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T16:36:56.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:56 smithi133 ceph-mon[32691]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T16:36:56.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:56 smithi116 ceph-mon[32712]: pgmap v295: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T16:36:57.472 INFO:teuthology.orchestra.run.smithi112.stdout:107374182460 2024-04-25T16:36:57.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T16:36:58.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:36:58 smithi133 ceph-mon[32691]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:36:58.487 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:36:58 smithi112 ceph-mon[29122]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:36:58.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:36:58 smithi116 ceph-mon[32712]: pgmap v296: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 -- 172.21.15.112:0/4193863453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac107c90 msgr2=0x7fd2ac1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 --2- 172.21.15.112:0/4193863453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac107c90 0x7fd2ac1080a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00099c0 tx=0x7fd2a0009cd0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 -- 172.21.15.112:0/4193863453 shutdown_connections 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 --2- 172.21.15.112:0/4193863453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ac1085e0 0x7fd2ac10ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 --2- 172.21.15.112:0/4193863453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2ac103360 0x7fd2ac103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 --2- 172.21.15.112:0/4193863453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac107c90 0x7fd2ac1080a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 -- 172.21.15.112:0/4193863453 >> 172.21.15.112:0/4193863453 conn(0x7fd2ac0fec90 msgr2=0x7fd2ac1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 -- 172.21.15.112:0/4193863453 shutdown_connections 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.736+0000 7fd2b194d700 1 -- 172.21.15.112:0/4193863453 wait complete. 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.737+0000 7fd2b194d700 1 Processor -- start 2024-04-25T16:36:58.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.737+0000 7fd2b194d700 1 -- start start 2024-04-25T16:36:58.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac103360 0x7fd2ac19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2ac107c90 0x7fd2ac19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ac1085e0 0x7fd2ac199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac113260 con 0x7fd2ac103360 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1130e0 con 0x7fd2ac107c90 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac1133e0 con 0x7fd2ac1085e0 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac103360 0x7fd2ac19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac103360 0x7fd2ac19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59616/0 (socket says 172.21.15.112:59616) 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 -- 172.21.15.112:0/2913710828 learned_addr learned my addr 172.21.15.112:0/2913710828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aa7fc700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2ac107c90 0x7fd2ac19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2ab7fe700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ac1085e0 0x7fd2ac199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 -- 172.21.15.112:0/2913710828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ac1085e0 msgr2=0x7fd2ac199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ac1085e0 0x7fd2ac199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 -- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2ac107c90 msgr2=0x7fd2ac19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:58.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2ac107c90 0x7fd2ac19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 -- 172.21.15.112:0/2913710828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac19a0f0 con 0x7fd2ac103360 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2ab7fe700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2ac1085e0 0x7fd2ac199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2aaffd700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac103360 0x7fd2ac19f820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00c920 tx=0x7fd29c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c007b00 con 0x7fd2ac103360 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29c004ce0 con 0x7fd2ac103360 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac19a3e0 con 0x7fd2ac103360 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c005550 con 0x7fd2ac103360 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.738+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1a66b0 con 0x7fd2ac103360 2024-04-25T16:36:58.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.739+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2ac063de0 con 0x7fd2ac103360 2024-04-25T16:36:58.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.741+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fd29c007ca0 con 0x7fd2ac103360 2024-04-25T16:36:58.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2b094b700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2940afc10 0x7fd2940b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:58.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd29c0ded90 con 0x7fd2ac103360 2024-04-25T16:36:58.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2aa7fc700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2940afc10 0x7fd2940b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2b094b700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] conn(0x7fd2940b9440 0x7fd2940bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 --> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2940bbf60 con 0x7fd2940b9440 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fd29c0df270 con 0x7fd2ac103360 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.742+0000 7fd2aa7fc700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2940afc10 0x7fd2940b2120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006010 tx=0x7fd2a0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.743+0000 7fd2ab7fe700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] conn(0x7fd2940b9440 0x7fd2940bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.743+0000 7fd2ab7fe700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] conn(0x7fd2940b9440 0x7fd2940bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:58.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.744+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== osd.7 v2:172.21.15.116:6816/590137148 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2940bbf60 con 0x7fd2940b9440 2024-04-25T16:36:58.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.760+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 --> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2ac064260 con 0x7fd2940b9440 2024-04-25T16:36:58.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.760+0000 7fd2b094b700 1 -- 172.21.15.112:0/2913710828 <== osd.7 v2:172.21.15.116:6816/590137148 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2ac064260 con 0x7fd2940b9440 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] conn(0x7fd2940b9440 msgr2=0x7fd2940bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] conn(0x7fd2940b9440 0x7fd2940bb890 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2940afc10 msgr2=0x7fd2940b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2940afc10 0x7fd2940b2120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006010 tx=0x7fd2a0000bc0 comp rx=0 tx=0).stop 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac103360 msgr2=0x7fd2ac19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 --2- 172.21.15.112:0/2913710828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2ac103360 0x7fd2ac19f820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00c920 tx=0x7fd29c00cc30 comp rx=0 tx=0).stop 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2ab7fe700 1 -- 172.21.15.112:0/2913710828 reap_dead start 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 shutdown_connections 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 >> 172.21.15.112:0/2913710828 conn(0x7fd2ac0fec90 msgr2=0x7fd2ac1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 shutdown_connections 2024-04-25T16:36:58.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:58.761+0000 7fd2b194d700 1 -- 172.21.15.112:0/2913710828 wait complete. 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 -- 172.21.15.112:0/4124179819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b441091d0 msgr2=0x7f3b441095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4124179819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b441091d0 0x7f3b441095e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009990 tx=0x7f3b34009ca0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 -- 172.21.15.112:0/4124179819 shutdown_connections 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4124179819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b44102fb0 0x7f3b44111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4124179819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b441091d0 0x7f3b441095e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4124179819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b44102660 0x7f3b44102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 -- 172.21.15.112:0/4124179819 >> 172.21.15.112:0/4124179819 conn(0x7f3b440fe3f0 msgr2=0x7f3b44100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 -- 172.21.15.112:0/4124179819 shutdown_connections 2024-04-25T16:36:59.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.386+0000 7f3b4999d700 1 -- 172.21.15.112:0/4124179819 wait complete. 2024-04-25T16:36:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 Processor -- start 2024-04-25T16:36:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 -- start start 2024-04-25T16:36:59.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b44102660 0x7f3b4419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b44102fb0 0x7f3b4419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b441091d0 0x7f3b4419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44114460 con 0x7f3b44102fb0 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441142e0 con 0x7f3b441091d0 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b4999d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3b441145e0 con 0x7f3b44102660 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b427fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b44102fb0 0x7f3b4419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b427fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b44102fb0 0x7f3b4419b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59630/0 (socket says 172.21.15.112:59630) 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b427fc700 1 -- 172.21.15.112:0/4087364755 learned_addr learned my addr 172.21.15.112:0/4087364755 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b437fe700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b441091d0 0x7f3b4419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.387+0000 7f3b42ffd700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b44102660 0x7f3b4419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b427fc700 1 -- 172.21.15.112:0/4087364755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b44102660 msgr2=0x7f3b4419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b427fc700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b44102660 0x7f3b4419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.388 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b427fc700 1 -- 172.21.15.112:0/4087364755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b441091d0 msgr2=0x7f3b4419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b427fc700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b441091d0 0x7f3b4419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b427fc700 1 -- 172.21.15.112:0/4087364755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4419fb90 con 0x7f3b44102fb0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b427fc700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b44102fb0 0x7f3b4419b310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c9e0 tx=0x7f3b3800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38007aa0 con 0x7f3b44102fb0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b38007c40 con 0x7f3b44102fb0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4419fe80 con 0x7f3b44102fb0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38018660 con 0x7f3b44102fb0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.388+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a03c0 con 0x7f3b44102fb0 2024-04-25T16:36:59.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.389+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b44063da0 con 0x7f3b44102fb0 2024-04-25T16:36:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.391+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f3b38018800 con 0x7f3b44102fb0 2024-04-25T16:36:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.391+0000 7f3b4899b700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3b2c0afc10 0x7f3b2c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:59.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.391+0000 7f3b42ffd700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3b2c0afc10 0x7f3b2c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.391+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f3b380df030 con 0x7f3b44102fb0 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.391+0000 7f3b4899b700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] conn(0x7f3b2c0b9440 0x7f3b2c0bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.392+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 --> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b3801fe50 con 0x7f3b2c0b9440 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.392+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f3b380df510 con 0x7f3b44102fb0 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.392+0000 7f3b437fe700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] conn(0x7f3b2c0b9440 0x7f3b2c0bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.392+0000 7f3b42ffd700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3b2c0afc10 0x7f3b2c0b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006010 tx=0x7f3b3401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.392+0000 7f3b437fe700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] conn(0x7f3b2c0b9440 0x7f3b2c0bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:36:59.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.393+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== osd.9 v2:172.21.15.112:6826/2630590642 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b3801fe50 con 0x7f3b2c0b9440 2024-04-25T16:36:59.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.410+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 --> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b440641a0 con 0x7f3b2c0b9440 2024-04-25T16:36:59.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.410+0000 7f3b4899b700 1 -- 172.21.15.112:0/4087364755 <== osd.9 v2:172.21.15.112:6826/2630590642 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b440641a0 con 0x7f3b2c0b9440 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] conn(0x7f3b2c0b9440 msgr2=0x7f3b2c0bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] conn(0x7f3b2c0b9440 0x7f3b2c0bb890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3b2c0afc10 msgr2=0x7f3b2c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f3b2c0afc10 0x7f3b2c0b2120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b34006010 tx=0x7f3b3401a040 comp rx=0 tx=0).stop 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b44102fb0 msgr2=0x7f3b4419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 --2- 172.21.15.112:0/4087364755 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b44102fb0 0x7f3b4419b310 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800c9e0 tx=0x7f3b3800ccf0 comp rx=0 tx=0).stop 2024-04-25T16:36:59.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b437fe700 1 -- 172.21.15.112:0/4087364755 reap_dead start 2024-04-25T16:36:59.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 shutdown_connections 2024-04-25T16:36:59.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 >> 172.21.15.112:0/4087364755 conn(0x7f3b440fe3f0 msgr2=0x7f3b44106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:36:59.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 shutdown_connections 2024-04-25T16:36:59.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:36:59.411+0000 7f3b4999d700 1 -- 172.21.15.112:0/4087364755 wait complete. 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 -- 172.21.15.112:0/3990401612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa570103360 msgr2=0x7fa570103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 --2- 172.21.15.112:0/3990401612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa570103360 0x7fa570103770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5600099c0 tx=0x7fa560009cd0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 -- 172.21.15.112:0/3990401612 shutdown_connections 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 --2- 172.21.15.112:0/3990401612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5701085e0 0x7fa57010ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 --2- 172.21.15.112:0/3990401612 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa570103360 0x7fa570103770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 --2- 172.21.15.112:0/3990401612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa570107c90 0x7fa5701080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 -- 172.21.15.112:0/3990401612 >> 172.21.15.112:0/3990401612 conn(0x7fa5700fec90 msgr2=0x7fa5701010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 -- 172.21.15.112:0/3990401612 shutdown_connections 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.039+0000 7fa5764a8700 1 -- 172.21.15.112:0/3990401612 wait complete. 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.040+0000 7fa5764a8700 1 Processor -- start 2024-04-25T16:37:00.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.040+0000 7fa5764a8700 1 -- start start 2024-04-25T16:37:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa570103360 0x7fa57019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa570107c90 0x7fa57019c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5701085e0 0x7fa57019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:00.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa570113290 con 0x7fa5701085e0 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa570113110 con 0x7fa570103360 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa570113410 con 0x7fa570107c90 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5701085e0 0x7fa57019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5701085e0 0x7fa57019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59640/0 (socket says 172.21.15.112:59640) 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 -- 172.21.15.112:0/2707839678 learned_addr learned my addr 172.21.15.112:0/2707839678 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa56f7fe700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa570107c90 0x7fa57019c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa56ffff700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa570103360 0x7fa57019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 -- 172.21.15.112:0/2707839678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa570107c90 msgr2=0x7fa57019c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa570107c90 0x7fa57019c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 -- 172.21.15.112:0/2707839678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa570103360 msgr2=0x7fa57019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa570103360 0x7fa57019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 -- 172.21.15.112:0/2707839678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5701a6ee0 con 0x7fa5701085e0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa574a45700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5701085e0 0x7fa57019c630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa56800b970 tx=0x7fa56800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56800d860 con 0x7fa5701085e0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa56800e430 con 0x7fa5701085e0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5701a71d0 con 0x7fa5701085e0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.041+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa568017660 con 0x7fa5701085e0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.042+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5701a76c0 con 0x7fa5701085e0 2024-04-25T16:37:00.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.042+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa570063da0 con 0x7fa5701085e0 2024-04-25T16:37:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.044+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fa56800da00 con 0x7fa5701085e0 2024-04-25T16:37:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56d7fa700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5580afc10 0x7fa5580b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56ffff700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5580afc10 0x7fa5580b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fa5680dee90 con 0x7fa5701085e0 2024-04-25T16:37:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56d7fa700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] conn(0x7fa5580b9440 0x7fa5580bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:00.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 --> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa568019e50 con 0x7fa5580b9440 2024-04-25T16:37:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fa5680df300 con 0x7fa5701085e0 2024-04-25T16:37:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56f7fe700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] conn(0x7fa5580b9440 0x7fa5580bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56ffff700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5580afc10 0x7fa5580b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa560009990 tx=0x7fa5600036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.045+0000 7fa56f7fe700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] conn(0x7fa5580b9440 0x7fa5580bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:00.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.046+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== osd.6 v2:172.21.15.112:6818/4240993708 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa568019e50 con 0x7fa5580b9440 2024-04-25T16:37:00.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.062+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 --> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5700641a0 con 0x7fa5580b9440 2024-04-25T16:37:00.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.062+0000 7fa56d7fa700 1 -- 172.21.15.112:0/2707839678 <== osd.6 v2:172.21.15.112:6818/4240993708 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5700641a0 con 0x7fa5580b9440 2024-04-25T16:37:00.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] conn(0x7fa5580b9440 msgr2=0x7fa5580bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] conn(0x7fa5580b9440 0x7fa5580bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5580afc10 msgr2=0x7fa5580b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5580afc10 0x7fa5580b2120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa560009990 tx=0x7fa5600036e0 comp rx=0 tx=0).stop 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5701085e0 msgr2=0x7fa57019c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 --2- 172.21.15.112:0/2707839678 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5701085e0 0x7fa57019c630 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa56800b970 tx=0x7fa56800bc80 comp rx=0 tx=0).stop 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa574a45700 1 -- 172.21.15.112:0/2707839678 reap_dead start 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 shutdown_connections 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 >> 172.21.15.112:0/2707839678 conn(0x7fa5700fec90 msgr2=0x7fa570101b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 shutdown_connections 2024-04-25T16:37:00.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.063+0000 7fa5764a8700 1 -- 172.21.15.112:0/2707839678 wait complete. 2024-04-25T16:37:00.296 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:00 smithi112 ceph-mon[29122]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:00.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:00 smithi133 ceph-mon[32691]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:00.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:00 smithi116 ceph-mon[32712]: pgmap v297: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:01.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/3970983478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e01085d0 msgr2=0x7fb8e010ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:01.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/3970983478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e01085d0 0x7fb8e010ff30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb8d800b0d0 tx=0x7fb8d800b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/3970983478 shutdown_connections 2024-04-25T16:37:01.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/3970983478 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e01085d0 0x7fb8e010ff30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/3970983478 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e0103350 0x7fb8e0103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/3970983478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8e0107c80 0x7fb8e0108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/3970983478 >> 172.21.15.112:0/3970983478 conn(0x7fb8e00fec30 msgr2=0x7fb8e0101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:01.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/3970983478 shutdown_connections 2024-04-25T16:37:01.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.358+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/3970983478 wait complete. 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 Processor -- start 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 -- start start 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0103350 0x7fb8e01a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8e0107c80 0x7fb8e01a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e01085d0 0x7fb8e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e0113220 con 0x7fb8e0103350 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01130a0 con 0x7fb8e0107c80 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e6e2d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e01133a0 con 0x7fb8e01085d0 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e4bc9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0103350 0x7fb8e01a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e4bc9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0103350 0x7fb8e01a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59674/0 (socket says 172.21.15.112:59674) 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.359+0000 7fb8e4bc9700 1 -- 172.21.15.112:0/77537878 learned_addr learned my addr 172.21.15.112:0/77537878 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8dffff700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8e0107c80 0x7fb8e01a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:01.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e53ca700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e01085d0 0x7fb8e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e4bc9700 1 -- 172.21.15.112:0/77537878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e01085d0 msgr2=0x7fb8e019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e4bc9700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8e01085d0 0x7fb8e019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e4bc9700 1 -- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8e0107c80 msgr2=0x7fb8e01a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e4bc9700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8e0107c80 0x7fb8e01a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e4bc9700 1 -- 172.21.15.112:0/77537878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e019b140 con 0x7fb8e0103350 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e4bc9700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0103350 0x7fb8e01a0870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400d980 tx=0x7fb8d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4017070 con 0x7fb8e0103350 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8d4005020 con 0x7fb8e0103350 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e019b430 con 0x7fb8e0103350 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400fa10 con 0x7fb8e0103350 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.360+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e01a7700 con 0x7fb8e0103350 2024-04-25T16:37:01.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.361+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb8e0063e20 con 0x7fb8e0103350 2024-04-25T16:37:01.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fb8d4004a30 con 0x7fb8e0103350 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8ddffb700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb8d00afc10 0x7fb8d00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8dffff700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb8d00afc10 0x7fb8d00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fb8d40df070 con 0x7fb8e0103350 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8ddffb700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] conn(0x7fb8d00b9440 0x7fb8d00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 --> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb8d00bbf60 con 0x7fb8d00b9440 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.363+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fb8d40df550 con 0x7fb8e0103350 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.364+0000 7fb8dffff700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb8d00afc10 0x7fb8d00b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0109730 tx=0x7fb8cc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.364+0000 7fb8e53ca700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] conn(0x7fb8d00b9440 0x7fb8d00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:01.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.364+0000 7fb8e53ca700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] conn(0x7fb8d00b9440 0x7fb8d00bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:01.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.365+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== osd.10 v2:172.21.15.116:6824/3599339690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb8d00bbf60 con 0x7fb8d00b9440 2024-04-25T16:37:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.381+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 --> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb8e00642a0 con 0x7fb8d00b9440 2024-04-25T16:37:01.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8ddffb700 1 -- 172.21.15.112:0/77537878 <== osd.10 v2:172.21.15.116:6824/3599339690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb8e00642a0 con 0x7fb8d00b9440 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] conn(0x7fb8d00b9440 msgr2=0x7fb8d00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] conn(0x7fb8d00b9440 0x7fb8d00bb890 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb8d00afc10 msgr2=0x7fb8d00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fb8d00afc10 0x7fb8d00b2120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0109730 tx=0x7fb8cc00a300 comp rx=0 tx=0).stop 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0103350 msgr2=0x7fb8e01a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 --2- 172.21.15.112:0/77537878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8e0103350 0x7fb8e01a0870 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400d980 tx=0x7fb8d400dc90 comp rx=0 tx=0).stop 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e53ca700 1 -- 172.21.15.112:0/77537878 reap_dead start 2024-04-25T16:37:01.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 shutdown_connections 2024-04-25T16:37:01.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 >> 172.21.15.112:0/77537878 conn(0x7fb8e00fec30 msgr2=0x7fb8e006c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:01.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 shutdown_connections 2024-04-25T16:37:01.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:01.382+0000 7fb8e6e2d700 1 -- 172.21.15.112:0/77537878 wait complete. 2024-04-25T16:37:02.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:02 smithi133 ceph-mon[32691]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:37:02.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:02 smithi116 ceph-mon[32712]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:37:02.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:02 smithi112 ceph-mon[29122]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1005389132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 msgr2=0x7f2fc8102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1005389132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc8102ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc009990 tx=0x7f2fbc009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1005389132 shutdown_connections 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1005389132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fc81031e0 0x7f2fc810f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1005389132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc81088c0 0x7f2fc8108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1005389132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc8102ca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1005389132 >> 172.21.15.112:0/1005389132 conn(0x7f2fc80fe3d0 msgr2=0x7f2fc8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1005389132 shutdown_connections 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1005389132 wait complete. 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.311+0000 7f2fcd28a700 1 Processor -- start 2024-04-25T16:37:04.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 -- start start 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc819ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc81031e0 0x7f2fc819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fc81088c0 0x7f2fc819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8112a40 con 0x7f2fc8102890 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc81128c0 con 0x7f2fc81031e0 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fcd28a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8112bc0 con 0x7f2fc81088c0 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc819ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc819ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59656/0 (socket says 172.21.15.112:59656) 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 -- 172.21.15.112:0/1036403230 learned_addr learned my addr 172.21.15.112:0/1036403230 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc67fc700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc81031e0 0x7f2fc819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc77fe700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fc81088c0 0x7f2fc819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 -- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fc81088c0 msgr2=0x7f2fc819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fc81088c0 0x7f2fc819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 -- 172.21.15.112:0/1036403230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc81031e0 msgr2=0x7f2fc819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc81031e0 0x7f2fc819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 -- 172.21.15.112:0/1036403230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc819fc00 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc67fc700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc81031e0 0x7f2fc819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2fc6ffd700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc819ad30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c8f0 tx=0x7f2fb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.312+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8007b00 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.313+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb8007ca0 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.313+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8018660 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.313+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc819fef0 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.313+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc81a7810 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.313+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2fc8063e20 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.315+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f2fb800f450 con 0x7f2fc8102890 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.315+0000 7f2faffff700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2fb00afd10 0x7f2fb00b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2fc67fc700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2fb00afd10 0x7f2fb00b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:04.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f2fb80def10 con 0x7f2fc8102890 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2faffff700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] conn(0x7f2fb00b9540 0x7f2fb00bb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 --> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2fb8015e70 con 0x7f2fb00b9540 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f2fb80df300 con 0x7f2fc8102890 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2fc67fc700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2fb00afd10 0x7f2fb00b2220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc006010 tx=0x7f2fbc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.316+0000 7f2fc77fe700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] conn(0x7f2fb00b9540 0x7f2fb00bb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.317+0000 7f2fc77fe700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] conn(0x7f2fb00b9540 0x7f2fb00bb990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.317+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== osd.11 v2:172.21.15.133:6824/2381034021 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2fb8015e70 con 0x7f2fb00b9540 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.334+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 --> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2fc8064220 con 0x7f2fb00b9540 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.334+0000 7f2faffff700 1 -- 172.21.15.112:0/1036403230 <== osd.11 v2:172.21.15.133:6824/2381034021 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2fc8064220 con 0x7f2fb00b9540 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] conn(0x7f2fb00b9540 msgr2=0x7f2fb00bb990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] conn(0x7f2fb00b9540 0x7f2fb00bb990 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2fb00afd10 msgr2=0x7f2fb00b2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2fb00afd10 0x7f2fb00b2220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc006010 tx=0x7f2fbc01a040 comp rx=0 tx=0).stop 2024-04-25T16:37:04.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 msgr2=0x7f2fc819ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 --2- 172.21.15.112:0/1036403230 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fc8102890 0x7f2fc819ad30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800c8f0 tx=0x7f2fb800ccb0 comp rx=0 tx=0).stop 2024-04-25T16:37:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fc77fe700 1 -- 172.21.15.112:0/1036403230 reap_dead start 2024-04-25T16:37:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 shutdown_connections 2024-04-25T16:37:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 >> 172.21.15.112:0/1036403230 conn(0x7f2fc80fe3d0 msgr2=0x7f2fc81009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 shutdown_connections 2024-04-25T16:37:04.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:00.335+0000 7f2fcd28a700 1 -- 172.21.15.112:0/1036403230 wait complete. 2024-04-25T16:37:04.331 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:04 smithi112 ceph-mon[29122]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T16:37:04.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:04 smithi133 ceph-mon[32691]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T16:37:04.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:04 smithi116 ceph-mon[32712]: pgmap v299: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T16:37:05.025 INFO:teuthology.orchestra.run.smithi112.stdout:300647710740 2024-04-25T16:37:05.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T16:37:05.536 INFO:teuthology.orchestra.run.smithi112.stdout:253403070495 2024-04-25T16:37:05.536 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T16:37:06.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:06 smithi112 ceph-mon[29122]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T16:37:06.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:06 smithi133 ceph-mon[32691]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T16:37:06.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:06 smithi116 ceph-mon[32712]: pgmap v300: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T16:37:06.704 INFO:teuthology.orchestra.run.smithi112.stdout:201863462953 2024-04-25T16:37:06.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T16:37:07.567 INFO:teuthology.orchestra.run.smithi112.stdout:274877906970 2024-04-25T16:37:07.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 -- 172.21.15.112:0/3835673426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8103070 msgr2=0x7fadf8111010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 --2- 172.21.15.112:0/3835673426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8103070 0x7fadf8111010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b0a0 tx=0x7fadf000b3b0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 -- 172.21.15.112:0/3835673426 shutdown_connections 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 --2- 172.21.15.112:0/3835673426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8103070 0x7fadf8111010 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 --2- 172.21.15.112:0/3835673426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf8109180 0x7fadf8109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 --2- 172.21.15.112:0/3835673426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadf8102720 0x7fadf8102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 -- 172.21.15.112:0/3835673426 >> 172.21.15.112:0/3835673426 conn(0x7fadf80fe370 msgr2=0x7fadf81007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 -- 172.21.15.112:0/3835673426 shutdown_connections 2024-04-25T16:37:07.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 -- 172.21.15.112:0/3835673426 wait complete. 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 Processor -- start 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.795+0000 7fadfe757700 1 -- start start 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8102720 0x7fadf81a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadf8103070 0x7fadf81a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf8109180 0x7fadf819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadf8114440 con 0x7fadf8102720 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81142c0 con 0x7fadf8109180 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81145c0 con 0x7fadf8103070 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8102720 0x7fadf81a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8102720 0x7fadf81a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45780/0 (socket says 172.21.15.112:45780) 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 -- 172.21.15.112:0/808412263 learned_addr learned my addr 172.21.15.112:0/808412263 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf77fe700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadf8103070 0x7fadf81a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:07.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfccf4700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf8109180 0x7fadf819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 -- 172.21.15.112:0/808412263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadf8103070 msgr2=0x7fadf81a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fadf8103070 0x7fadf81a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 -- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf8109180 msgr2=0x7fadf819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf8109180 0x7fadf819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 -- 172.21.15.112:0/808412263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf819b210 con 0x7fadf8102720 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfccf4700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf8109180 0x7fadf819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:37:07.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf7fff700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8102720 0x7fadf81a08c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fade800c9e0 tx=0x7fade800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8007aa0 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf819b500 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fade8007c40 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.796+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8018660 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.797+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81a7770 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.797+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadf8063da0 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.799+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fade8018800 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf57fa700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fade00afc10 0x7fade00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf77fe700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fade00afc10 0x7fade00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fade80df260 con 0x7fadf8102720 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf57fa700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] conn(0x7fade00b9440 0x7fade00bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:07.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 --> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fade801fe50 con 0x7fade00b9440 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fade80df6c0 con 0x7fadf8102720 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadf77fe700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fade00afc10 0x7fade00b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadf806b990 tx=0x7fadec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.800+0000 7fadfccf4700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] conn(0x7fade00b9440 0x7fade00bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.801+0000 7fadfccf4700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] conn(0x7fade00b9440 0x7fade00bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.802+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== osd.1 v2:172.21.15.116:6800/2666280069 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fade801fe50 con 0x7fade00b9440 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.818+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 --> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadf80641a0 con 0x7fade00b9440 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.818+0000 7fadf57fa700 1 -- 172.21.15.112:0/808412263 <== osd.1 v2:172.21.15.116:6800/2666280069 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fadf80641a0 con 0x7fade00b9440 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.818+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] conn(0x7fade00b9440 msgr2=0x7fade00bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.818+0000 7fadfe757700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] conn(0x7fade00b9440 0x7fade00bb890 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fade00afc10 msgr2=0x7fade00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fade00afc10 0x7fade00b2120 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadf806b990 tx=0x7fadec00b410 comp rx=0 tx=0).stop 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8102720 msgr2=0x7fadf81a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 --2- 172.21.15.112:0/808412263 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf8102720 0x7fadf81a08c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fade800c9e0 tx=0x7fade800ccf0 comp rx=0 tx=0).stop 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfccf4700 1 -- 172.21.15.112:0/808412263 reap_dead start 2024-04-25T16:37:07.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 shutdown_connections 2024-04-25T16:37:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 >> 172.21.15.112:0/808412263 conn(0x7fadf80fe370 msgr2=0x7fadf80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 shutdown_connections 2024-04-25T16:37:07.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:07.819+0000 7fadfe757700 1 -- 172.21.15.112:0/808412263 wait complete. 2024-04-25T16:37:08.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 -- 172.21.15.112:0/1486507544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 msgr2=0x7faad410ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:08.334 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 --2- 172.21.15.112:0/1486507544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad410ff70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faacc0099c0 tx=0x7faacc009cd0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 -- 172.21.15.112:0/1486507544 shutdown_connections 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 --2- 172.21.15.112:0/1486507544 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad410ff70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 --2- 172.21.15.112:0/1486507544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad4103390 0x7faad41037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 --2- 172.21.15.112:0/1486507544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faad4107cc0 0x7faad41080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 -- 172.21.15.112:0/1486507544 >> 172.21.15.112:0/1486507544 conn(0x7faad40fec90 msgr2=0x7faad41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 -- 172.21.15.112:0/1486507544 shutdown_connections 2024-04-25T16:37:08.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 -- 172.21.15.112:0/1486507544 wait complete. 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.336+0000 7faadb408700 1 Processor -- start 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 -- start start 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faad4103390 0x7faad419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad4107cc0 0x7faad419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7faad41132b0 con 0x7faad4108610 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faad4113130 con 0x7faad4103390 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faadb408700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faad4113430 con 0x7faad4107cc0 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:08.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad419f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45810/0 (socket says 172.21.15.112:45810) 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 -- 172.21.15.112:0/2519888051 learned_addr learned my addr 172.21.15.112:0/2519888051 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad89a3700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad4107cc0 0x7faad419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad91a4700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faad4103390 0x7faad419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 -- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad4107cc0 msgr2=0x7faad419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faad4107cc0 0x7faad419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 -- 172.21.15.112:0/2519888051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faad4103390 msgr2=0x7faad419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faad4103390 0x7faad419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 -- 172.21.15.112:0/2519888051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad419fc40 con 0x7faad4108610 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.337+0000 7faad99a5700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad419f5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faacc009990 tx=0x7faacc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.338+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc027070 con 0x7faad4108610 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.338+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad419fed0 con 0x7faad4108610 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.338+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faacc004150 con 0x7faad4108610 2024-04-25T16:37:08.338 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.338+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faacc00f460 con 0x7faad4108610 2024-04-25T16:37:08.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.338+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad41a03b0 con 0x7faad4108610 2024-04-25T16:37:08.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.338+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faad4101580 con 0x7faad4108610 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.340+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7faacc02a070 con 0x7faad4108610 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad27fc700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7faabc0afc10 0x7faabc0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad91a4700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7faabc0afc10 0x7faabc0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7faacc0e0740 con 0x7faad4108610 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad27fc700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] conn(0x7faabc0b9310 0x7faabc0bb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 --> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faacc00fe50 con 0x7faabc0b9310 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7faacc0e0ba0 con 0x7faad4108610 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad91a4700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7faabc0afc10 0x7faabc0b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faac40060b0 tx=0x7faac4007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:08.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.341+0000 7faad89a3700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] conn(0x7faabc0b9310 0x7faabc0bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:08.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.342+0000 7faad89a3700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] conn(0x7faabc0b9310 0x7faabc0bb760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:08.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.343+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== osd.8 v2:172.21.15.133:6816/1430124311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faacc00fe50 con 0x7faabc0b9310 2024-04-25T16:37:08.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.359+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 --> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faad4104840 con 0x7faabc0b9310 2024-04-25T16:37:08.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.359+0000 7faad27fc700 1 -- 172.21.15.112:0/2519888051 <== osd.8 v2:172.21.15.133:6816/1430124311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faad4104840 con 0x7faabc0b9310 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] conn(0x7faabc0b9310 msgr2=0x7faabc0bb760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] conn(0x7faabc0b9310 0x7faabc0bb760 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7faabc0afc10 msgr2=0x7faabc0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7faabc0afc10 0x7faabc0b2120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faac40060b0 tx=0x7faac4007b80 comp rx=0 tx=0).stop 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 msgr2=0x7faad419f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 --2- 172.21.15.112:0/2519888051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faad4108610 0x7faad419f5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faacc009990 tx=0x7faacc0047b0 comp rx=0 tx=0).stop 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faad99a5700 1 -- 172.21.15.112:0/2519888051 reap_dead start 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 shutdown_connections 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 >> 172.21.15.112:0/2519888051 conn(0x7faad40fec90 msgr2=0x7faad4107430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 shutdown_connections 2024-04-25T16:37:08.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:08.360+0000 7faadb408700 1 -- 172.21.15.112:0/2519888051 wait complete. 2024-04-25T16:37:08.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:08 smithi133 ceph-mon[32691]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:08.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:08 smithi116 ceph-mon[32712]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:08.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:08 smithi112 ceph-mon[29122]: pgmap v301: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:09.484 INFO:teuthology.orchestra.run.smithi112.stdout:180388626479 2024-04-25T16:37:09.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T16:37:10.021 INFO:teuthology.orchestra.run.smithi112.stdout:227633266725 2024-04-25T16:37:10.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T16:37:10.288 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:10 smithi112 ceph-mon[29122]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T16:37:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:10 smithi133 ceph-mon[32691]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T16:37:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:10 smithi116 ceph-mon[32712]: pgmap v302: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T16:37:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 -- 172.21.15.112:0/2236688996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c104480 msgr2=0x7f5b1c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2236688996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c104480 0x7f5b1c104890 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400b0d0 tx=0x7f5b1400b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 -- 172.21.15.112:0/2236688996 shutdown_connections 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2236688996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b1c109700 0x7f5b1c111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2236688996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c104480 0x7f5b1c104890 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2236688996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b1c108db0 0x7f5b1c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 -- 172.21.15.112:0/2236688996 >> 172.21.15.112:0/2236688996 conn(0x7f5b1c0ffd70 msgr2=0x7f5b1c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 -- 172.21.15.112:0/2236688996 shutdown_connections 2024-04-25T16:37:10.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.605+0000 7f5b20dba700 1 -- 172.21.15.112:0/2236688996 wait complete. 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.606+0000 7f5b20dba700 1 Processor -- start 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.606+0000 7f5b20dba700 1 -- start start 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b20dba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b1c104480 0x7f5b1c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b20dba700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c108db0 0x7f5b1c1a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b20dba700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b1c109700 0x7f5b1c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b20dba700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c114330 con 0x7f5b1c108db0 2024-04-25T16:37:10.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b20dba700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1141b0 con 0x7f5b1c104480 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b20dba700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c1144b0 con 0x7f5b1c109700 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c108db0 0x7f5b1c1a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c108db0 0x7f5b1c1a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45828/0 (socket says 172.21.15.112:45828) 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 -- 172.21.15.112:0/2756475530 learned_addr learned my addr 172.21.15.112:0/2756475530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b1a59c700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b1c104480 0x7f5b1c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 -- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b1c109700 msgr2=0x7f5b1c19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b1ad9d700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b1c109700 0x7f5b1c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b1c109700 0x7f5b1c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 -- 172.21.15.112:0/2756475530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b1c104480 msgr2=0x7f5b1c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b1c104480 0x7f5b1c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 -- 172.21.15.112:0/2756475530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c19b160 con 0x7f5b1c108db0 2024-04-25T16:37:10.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.607+0000 7f5b19d9b700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c108db0 0x7f5b1c1a0dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400d980 tx=0x7f5b0400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.608+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b04017070 con 0x7f5b1c108db0 2024-04-25T16:37:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.608+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b04010460 con 0x7f5b1c108db0 2024-04-25T16:37:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.608+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c19b450 con 0x7f5b1c108db0 2024-04-25T16:37:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.608+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400de60 con 0x7f5b1c108db0 2024-04-25T16:37:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.608+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1a7720 con 0x7f5b1c108db0 2024-04-25T16:37:10.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.608+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5b1c063e20 con 0x7f5b1c108db0 2024-04-25T16:37:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.610+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f5b04010610 con 0x7f5b1c108db0 2024-04-25T16:37:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b137fe700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5b080afc10 0x7f5b080b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b1a59c700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5b080afc10 0x7f5b080b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5b040ded70 con 0x7f5b1c108db0 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b137fe700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] conn(0x7f5b080b9440 0x7f5b080bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 --> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5b0401ce50 con 0x7f5b080b9440 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f5b040df250 con 0x7f5b1c108db0 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b1a59c700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5b080afc10 0x7f5b080b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009900 tx=0x7f5b1400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.611+0000 7f5b1ad9d700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] conn(0x7f5b080b9440 0x7f5b080bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.612+0000 7f5b1ad9d700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] conn(0x7f5b080b9440 0x7f5b080bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:10.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.613+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== osd.2 v2:172.21.15.133:6800/1571915976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5b0401ce50 con 0x7f5b080b9440 2024-04-25T16:37:10.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.629+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 --> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5b1c0642a0 con 0x7f5b080b9440 2024-04-25T16:37:10.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.629+0000 7f5b137fe700 1 -- 172.21.15.112:0/2756475530 <== osd.2 v2:172.21.15.133:6800/1571915976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5b1c0642a0 con 0x7f5b080b9440 2024-04-25T16:37:10.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] conn(0x7f5b080b9440 msgr2=0x7f5b080bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] conn(0x7f5b080b9440 0x7f5b080bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5b080afc10 msgr2=0x7f5b080b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5b080afc10 0x7f5b080b2120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009900 tx=0x7f5b1400b040 comp rx=0 tx=0).stop 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c108db0 msgr2=0x7f5b1c1a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 --2- 172.21.15.112:0/2756475530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b1c108db0 0x7f5b1c1a0dd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400d980 tx=0x7f5b0400dd40 comp rx=0 tx=0).stop 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b1ad9d700 1 -- 172.21.15.112:0/2756475530 reap_dead start 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 shutdown_connections 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 >> 172.21.15.112:0/2756475530 conn(0x7f5b1c0ffd70 msgr2=0x7f5b1c108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 shutdown_connections 2024-04-25T16:37:10.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.630+0000 7f5b20dba700 1 -- 172.21.15.112:0/2756475530 wait complete. 2024-04-25T16:37:10.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 -- 172.21.15.112:0/2737053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c109700 msgr2=0x7f900c111060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 --2- 172.21.15.112:0/2737053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c109700 0x7f900c111060 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f900400b0d0 tx=0x7f900400b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 -- 172.21.15.112:0/2737053292 shutdown_connections 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 --2- 172.21.15.112:0/2737053292 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c109700 0x7f900c111060 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 --2- 172.21.15.112:0/2737053292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f900c104480 0x7f900c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 --2- 172.21.15.112:0/2737053292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c108db0 0x7f900c1091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 -- 172.21.15.112:0/2737053292 >> 172.21.15.112:0/2737053292 conn(0x7f900c0ffd70 msgr2=0x7f900c1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 -- 172.21.15.112:0/2737053292 shutdown_connections 2024-04-25T16:37:10.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.814+0000 7f9013656700 1 -- 172.21.15.112:0/2737053292 wait complete. 2024-04-25T16:37:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 Processor -- start 2024-04-25T16:37:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 -- start start 2024-04-25T16:37:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f900c104480 0x7f900c1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c108db0 0x7f900c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c109700 0x7f900c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f900c114330 con 0x7f900c108db0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1141b0 con 0x7f900c109700 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9013656700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1144b0 con 0x7f900c104480 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9010bf1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c108db0 0x7f900c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9010bf1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c108db0 0x7f900c1a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45842/0 (socket says 172.21.15.112:45842) 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9010bf1700 1 -- 172.21.15.112:0/2246060257 learned_addr learned my addr 172.21.15.112:0/2246060257 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f90113f2700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f900c104480 0x7f900c1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.815+0000 7f9011bf3700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c109700 0x7f900c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9010bf1700 1 -- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f900c104480 msgr2=0x7f900c1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9010bf1700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f900c104480 0x7f900c1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9010bf1700 1 -- 172.21.15.112:0/2246060257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c109700 msgr2=0x7f900c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9010bf1700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c109700 0x7f900c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9010bf1700 1 -- 172.21.15.112:0/2246060257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c19b1b0 con 0x7f900c108db0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9010bf1700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c108db0 0x7f900c1a0df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f900000d980 tx=0x7f900000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9000017070 con 0x7f900c108db0 2024-04-25T16:37:10.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9000009de0 con 0x7f900c108db0 2024-04-25T16:37:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c19b4a0 con 0x7f900c108db0 2024-04-25T16:37:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900000f3c0 con 0x7f900c108db0 2024-04-25T16:37:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.816+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1a7740 con 0x7f900c108db0 2024-04-25T16:37:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.817+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f900c063da0 con 0x7f900c108db0 2024-04-25T16:37:10.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.819+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f9000009930 con 0x7f900c108db0 2024-04-25T16:37:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.819+0000 7f8ffe7fc700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ff80afc90 0x7f8ff80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.819+0000 7f90113f2700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ff80afc90 0x7f8ff80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.819+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f90000de9c0 con 0x7f900c108db0 2024-04-25T16:37:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.820+0000 7f8ffe7fc700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] conn(0x7f8ff80b94c0 0x7f8ff80bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.820+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 --> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9000011e50 con 0x7f8ff80b94c0 2024-04-25T16:37:10.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.820+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f90000deea0 con 0x7f900c108db0 2024-04-25T16:37:10.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.820+0000 7f9011bf3700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] conn(0x7f8ff80b94c0 0x7f8ff80bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:10.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.820+0000 7f90113f2700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ff80afc90 0x7f8ff80b21a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9008005950 tx=0x7f900800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:10.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.820+0000 7f9011bf3700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] conn(0x7f8ff80b94c0 0x7f8ff80bb910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:10.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.821+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== osd.5 v2:172.21.15.133:6808/3110277631 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9000011e50 con 0x7f8ff80b94c0 2024-04-25T16:37:10.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.844+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 --> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f900c0641a0 con 0x7f8ff80b94c0 2024-04-25T16:37:10.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f8ffe7fc700 1 -- 172.21.15.112:0/2246060257 <== osd.5 v2:172.21.15.133:6808/3110277631 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f900c0641a0 con 0x7f8ff80b94c0 2024-04-25T16:37:10.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] conn(0x7f8ff80b94c0 msgr2=0x7f8ff80bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] conn(0x7f8ff80b94c0 0x7f8ff80bb910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ff80afc90 msgr2=0x7f8ff80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8ff80afc90 0x7f8ff80b21a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9008005950 tx=0x7f900800a300 comp rx=0 tx=0).stop 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c108db0 msgr2=0x7f900c1a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 --2- 172.21.15.112:0/2246060257 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f900c108db0 0x7f900c1a0df0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f900000d980 tx=0x7f900000dc90 comp rx=0 tx=0).stop 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9011bf3700 1 -- 172.21.15.112:0/2246060257 reap_dead start 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 shutdown_connections 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 >> 172.21.15.112:0/2246060257 conn(0x7f900c0ffd70 msgr2=0x7f900c1084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.845+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 shutdown_connections 2024-04-25T16:37:10.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:10.846+0000 7f9013656700 1 -- 172.21.15.112:0/2246060257 wait complete. 2024-04-25T16:37:11.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 -- 172.21.15.112:0/53227039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 msgr2=0x7f5ccc103760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 --2- 172.21.15.112:0/53227039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc103760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00099c0 tx=0x7f5cc0009cd0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 -- 172.21.15.112:0/53227039 shutdown_connections 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 --2- 172.21.15.112:0/53227039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1085d0 0x7f5ccc10ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 --2- 172.21.15.112:0/53227039 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc103760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 --2- 172.21.15.112:0/53227039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ccc107c80 0x7f5ccc108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 -- 172.21.15.112:0/53227039 >> 172.21.15.112:0/53227039 conn(0x7f5ccc0fec30 msgr2=0x7f5ccc101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.275+0000 7f5cd1cef700 1 -- 172.21.15.112:0/53227039 shutdown_connections 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.276+0000 7f5cd1cef700 1 -- 172.21.15.112:0/53227039 wait complete. 2024-04-25T16:37:11.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.276+0000 7f5cd1cef700 1 Processor -- start 2024-04-25T16:37:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.276+0000 7f5cd1cef700 1 -- start start 2024-04-25T16:37:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5cd1cef700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc0ff110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5cd1cef700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ccc107c80 0x7f5ccc0ff650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5cd1cef700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1085d0 0x7f5ccc0ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5cd1cef700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc113200 con 0x7f5ccc103350 2024-04-25T16:37:11.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5cd1cef700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc113080 con 0x7f5ccc1085d0 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5cd1cef700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc113380 con 0x7f5ccc107c80 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc0ff110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc0ff110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45860/0 (socket says 172.21.15.112:45860) 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 -- 172.21.15.112:0/279631171 learned_addr learned my addr 172.21.15.112:0/279631171 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccbfff700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1085d0 0x7f5ccc0ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccaffd700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ccc107c80 0x7f5ccc0ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 -- 172.21.15.112:0/279631171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ccc107c80 msgr2=0x7f5ccc0ff650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ccc107c80 0x7f5ccc0ff650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 -- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1085d0 msgr2=0x7f5ccc0ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1085d0 0x7f5ccc0ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 -- 172.21.15.112:0/279631171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc1a90a0 con 0x7f5ccc103350 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.277+0000 7f5ccb7fe700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc0ff110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000b580 tx=0x7f5cc00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:11.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.278+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0027070 con 0x7f5ccc103350 2024-04-25T16:37:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.278+0000 7f5cd1cef700 1 -- 172.21.15.112:0/279631171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc1a9330 con 0x7f5ccc103350 2024-04-25T16:37:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.278+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cc0003f20 con 0x7f5ccc103350 2024-04-25T16:37:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.278+0000 7f5cd1cef700 1 -- 172.21.15.112:0/279631171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc1a97e0 con 0x7f5ccc103350 2024-04-25T16:37:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.278+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0022d10 con 0x7f5ccc103350 2024-04-25T16:37:11.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.279+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5cac001030 con 0x7f5ccc103350 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.280+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f5cc0022470 con 0x7f5ccc103350 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.281+0000 7f5cc8ff9700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5cb40afc10 0x7f5cb40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.281+0000 7f5ccaffd700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5cb40afc10 0x7f5cb40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.281+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5cc00e0070 con 0x7f5ccc103350 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.281+0000 7f5cc8ff9700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] conn(0x7f5cb40b9440 0x7f5cb40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.281+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 --> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5cc00043d0 con 0x7f5cb40b9440 2024-04-25T16:37:11.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.281+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f5cc00e0550 con 0x7f5ccc103350 2024-04-25T16:37:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.282+0000 7f5ccaffd700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5cb40afc10 0x7f5cb40b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc1079b0 tx=0x7f5cbc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.282+0000 7f5ccbfff700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] conn(0x7f5cb40b9440 0x7f5cb40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.282+0000 7f5ccbfff700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] conn(0x7f5cb40b9440 0x7f5cb40bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:11.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.283+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== osd.4 v2:172.21.15.116:6808/2551746095 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5cc00043d0 con 0x7f5cb40b9440 2024-04-25T16:37:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 --> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5cac002e60 con 0x7f5cb40b9440 2024-04-25T16:37:11.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cc8ff9700 1 -- 172.21.15.112:0/279631171 <== osd.4 v2:172.21.15.116:6808/2551746095 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5cac002e60 con 0x7f5cb40b9440 2024-04-25T16:37:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] conn(0x7f5cb40b9440 msgr2=0x7f5cb40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:11.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] conn(0x7f5cb40b9440 0x7f5cb40bb890 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5cb40afc10 msgr2=0x7f5cb40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f5cb40afc10 0x7f5cb40b2120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc1079b0 tx=0x7f5cbc00b410 comp rx=0 tx=0).stop 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 msgr2=0x7f5ccc0ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.300+0000 7f5cb27fc700 1 --2- 172.21.15.112:0/279631171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ccc103350 0x7f5ccc0ff110 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5cc000b580 tx=0x7f5cc00047b0 comp rx=0 tx=0).stop 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.301+0000 7f5ccbfff700 1 -- 172.21.15.112:0/279631171 reap_dead start 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.301+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 shutdown_connections 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.301+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 >> 172.21.15.112:0/279631171 conn(0x7f5ccc0fec30 msgr2=0x7f5ccc106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.301+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 shutdown_connections 2024-04-25T16:37:11.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:11.301+0000 7f5cb27fc700 1 -- 172.21.15.112:0/279631171 wait complete. 2024-04-25T16:37:11.902 INFO:teuthology.orchestra.run.smithi112.stdout:154618822710 2024-04-25T16:37:11.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T16:37:12.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:12 smithi112 ceph-mon[29122]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:12.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:12 smithi133 ceph-mon[32691]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:12.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:12 smithi116 ceph-mon[32712]: pgmap v303: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:12.862 INFO:teuthology.orchestra.run.smithi112.stdout:77309411397 2024-04-25T16:37:12.862 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T16:37:13.260 INFO:teuthology.orchestra.run.smithi112.stdout:55834574922 2024-04-25T16:37:13.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T16:37:14.020 INFO:teuthology.orchestra.run.smithi112.stdout:133143986235 2024-04-25T16:37:14.021 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T16:37:14.021 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:14.042 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:14.049 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:14.057 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:14.311 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:14 smithi112 ceph-mon[29122]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:14.315 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:14.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:14 smithi133 ceph-mon[32691]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:14.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:14 smithi116 ceph-mon[32712]: pgmap v304: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T16:37:14.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/660804741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 msgr2=0x7f7cec103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/660804741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec103c60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009990 tx=0x7f7ce0009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/660804741 shutdown_connections 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/660804741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cec1041a0 0x7f7cec10ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/660804741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cec1080b0 0x7f7cec1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/660804741 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec103c60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/660804741 >> 172.21.15.112:0/660804741 conn(0x7f7cec0fe3f0 msgr2=0x7f7cec100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/660804741 shutdown_connections 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.809+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/660804741 wait complete. 2024-04-25T16:37:14.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.810+0000 7f7cf1f3d700 1 Processor -- start 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.810+0000 7f7cf1f3d700 1 -- start start 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cec1041a0 0x7f7cec1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cec1080b0 0x7f7cec19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec113250 con 0x7f7cec103850 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec1130d0 con 0x7f7cec1080b0 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec1133d0 con 0x7f7cec1041a0 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45888/0 (socket says 172.21.15.112:45888) 2024-04-25T16:37:14.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 -- 172.21.15.112:0/3581139032 learned_addr learned my addr 172.21.15.112:0/3581139032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceaffd700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cec1041a0 0x7f7cec1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cebfff700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cec1080b0 0x7f7cec19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 -- 172.21.15.112:0/3581139032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cec1041a0 msgr2=0x7f7cec1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cec1041a0 0x7f7cec1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 -- 172.21.15.112:0/3581139032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cec1080b0 msgr2=0x7f7cec19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cec1080b0 0x7f7cec19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 -- 172.21.15.112:0/3581139032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec19b110 con 0x7f7cec103850 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceaffd700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cec1041a0 0x7f7cec1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ceb7fe700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec1a0840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4007ee0 tx=0x7f7ce400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce401a070 con 0x7f7cec103850 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce400ddd0 con 0x7f7cec103850 2024-04-25T16:37:14.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec19b400 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce40126f0 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.811+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec1a76d0 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.812+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7cec063e20 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.814+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f7ce401d070 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.814+0000 7f7ce8ff9700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7cd40afc10 0x7f7cd40b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7ceaffd700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7cd40afc10 0x7f7cd40b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f7ce40df240 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7ce8ff9700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] conn(0x7f7cd40b9440 0x7f7cd40bb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 --> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cd40bbf60 con 0x7f7cd40b9440 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f7ce40df6a0 con 0x7f7cec103850 2024-04-25T16:37:14.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7cebfff700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] conn(0x7f7cd40b9440 0x7f7cd40bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7ceaffd700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7cd40afc10 0x7f7cd40b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0006010 tx=0x7f7ce001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.815+0000 7f7cebfff700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] conn(0x7f7cd40b9440 0x7f7cd40bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:14.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.816+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== osd.0 v2:172.21.15.112:6802/2222102236 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7cd40bbf60 con 0x7f7cd40b9440 2024-04-25T16:37:14.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 --> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7cec0642a0 con 0x7f7cd40b9440 2024-04-25T16:37:14.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7ce8ff9700 1 -- 172.21.15.112:0/3581139032 <== osd.0 v2:172.21.15.112:6802/2222102236 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7cec0642a0 con 0x7f7cd40b9440 2024-04-25T16:37:14.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] conn(0x7f7cd40b9440 msgr2=0x7f7cd40bb890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:14.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] conn(0x7f7cd40b9440 0x7f7cd40bb890 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7cd40afc10 msgr2=0x7f7cd40b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7cd40afc10 0x7f7cd40b2120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0006010 tx=0x7f7ce001a040 comp rx=0 tx=0).stop 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 msgr2=0x7f7cec1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cf1f3d700 1 --2- 172.21.15.112:0/3581139032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7cec103850 0x7f7cec1a0840 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4007ee0 tx=0x7f7ce400d3b0 comp rx=0 tx=0).stop 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.836+0000 7f7cebfff700 1 -- 172.21.15.112:0/3581139032 reap_dead start 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.837+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 shutdown_connections 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.837+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 >> 172.21.15.112:0/3581139032 conn(0x7f7cec0fe3f0 msgr2=0x7f7cec1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.837+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 shutdown_connections 2024-04-25T16:37:14.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:14.837+0000 7f7cf1f3d700 1 -- 172.21.15.112:0/3581139032 wait complete. 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T16:37:15.470 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T16:37:15.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-04-25T16:37:15.471 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:15 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-04-25T16:37:15.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T16:37:15.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-04-25T16:37:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:15 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-04-25T16:37:15.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]: dispatch 2024-04-25T16:37:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:15 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-04-25T16:37:16.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T16:37:16.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-04-25T16:37:16.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:16 smithi112 ceph-mon[29122]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T16:37:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T16:37:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-04-25T16:37:16.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:16 smithi116 ceph-mon[32712]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T16:37:16.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: pgmap v305: 97 pgs: 97 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 8]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-04-25T16:37:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:16 smithi133 ceph-mon[32691]: osdmap e76: 12 total, 12 up, 12 in 2024-04-25T16:37:16.810 INFO:teuthology.orchestra.run.smithi112.stdout:34359738448 2024-04-25T16:37:16.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T16:37:17.384 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:17 smithi112 ceph-mon[29122]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T16:37:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:17 smithi116 ceph-mon[32712]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T16:37:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:17 smithi133 ceph-mon[32691]: osdmap e77: 12 total, 12 up, 12 in 2024-04-25T16:37:18.131 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:18.462 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:18 smithi112 ceph-mon[29122]: pgmap v308: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T16:37:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:18 smithi116 ceph-mon[32712]: pgmap v308: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T16:37:18.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:18 smithi133 ceph-mon[32691]: pgmap v308: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T16:37:20.371 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:20.372 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:20.477 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:20 smithi112 ceph-mon[29122]: pgmap v309: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T16:37:20.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:20 smithi116 ceph-mon[32712]: pgmap v309: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T16:37:20.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 -- 172.21.15.112:0/633746758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 msgr2=0x7fa6881080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 --2- 172.21.15.112:0/633746758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa6881080d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa67c0099c0 tx=0x7fa67c009cd0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 -- 172.21.15.112:0/633746758 shutdown_connections 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 --2- 172.21.15.112:0/633746758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688108610 0x7fa68810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 --2- 172.21.15.112:0/633746758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa688103390 0x7fa6881037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 --2- 172.21.15.112:0/633746758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa6881080d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 -- 172.21.15.112:0/633746758 >> 172.21.15.112:0/633746758 conn(0x7fa6880fec90 msgr2=0x7fa6881010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 -- 172.21.15.112:0/633746758 shutdown_connections 2024-04-25T16:37:20.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.594+0000 7fa68e05a700 1 -- 172.21.15.112:0/633746758 wait complete. 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 Processor -- start 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 -- start start 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688103390 0x7fa68819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa68819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa688108610 0x7fa68819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa688113270 con 0x7fa688107cc0 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881130f0 con 0x7fa688108610 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa68e05a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa6881133f0 con 0x7fa688103390 2024-04-25T16:37:20.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa686ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa68819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa686ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa68819c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55980/0 (socket says 172.21.15.112:55980) 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa686ffd700 1 -- 172.21.15.112:0/1101750240 learned_addr learned my addr 172.21.15.112:0/1101750240 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.595+0000 7fa687fff700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa688108610 0x7fa68819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa6877fe700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688103390 0x7fa68819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa686ffd700 1 -- 172.21.15.112:0/1101750240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688103390 msgr2=0x7fa68819bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa686ffd700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688103390 0x7fa68819bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa686ffd700 1 -- 172.21.15.112:0/1101750240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa688108610 msgr2=0x7fa68819c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa686ffd700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa688108610 0x7fa68819c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa686ffd700 1 -- 172.21.15.112:0/1101750240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6881a6f00 con 0x7fa688107cc0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa686ffd700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa68819c0f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005b40 tx=0x7fa67c0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c026070 con 0x7fa688107cc0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6881a7190 con 0x7fa688107cc0 2024-04-25T16:37:20.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67c00f460 con 0x7fa688107cc0 2024-04-25T16:37:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67c009e60 con 0x7fa688107cc0 2024-04-25T16:37:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.596+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881a7640 con 0x7fa688107cc0 2024-04-25T16:37:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.598+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa688051190 con 0x7fa688107cc0 2024-04-25T16:37:20.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.599+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fa67c029070 con 0x7fa688107cc0 2024-04-25T16:37:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.600+0000 7fa684ff9700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa6700afc10 0x7fa6700b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.600+0000 7fa6877fe700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa6700afc10 0x7fa6700b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:20.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.600+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa67c0df650 con 0x7fa688107cc0 2024-04-25T16:37:20.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.600+0000 7fa6877fe700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa6700afc10 0x7fa6700b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa678009bf0 tx=0x7fa678009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:20.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.605+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa67c013070 con 0x7fa688107cc0 2024-04-25T16:37:20.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:20 smithi133 ceph-mon[32691]: pgmap v309: 97 pgs: 4 peering, 93 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-25T16:37:20.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.813+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fa68802e510 con 0x7fa688107cc0 2024-04-25T16:37:20.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.813+0000 7fa684ff9700 1 -- 172.21.15.112:0/1101750240 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa67c0a8140 con 0x7fa688107cc0 2024-04-25T16:37:20.812 INFO:teuthology.orchestra.run.smithi112.stdout:300647710744 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa6700afc10 msgr2=0x7fa6700b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa6700afc10 0x7fa6700b2120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa678009bf0 tx=0x7fa678009260 comp rx=0 tx=0).stop 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 msgr2=0x7fa68819c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa68819c0f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa67c005b40 tx=0x7fa67c0055f0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 shutdown_connections 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa6700afc10 0x7fa6700b2120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa688108610 0x7fa68819c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa688103390 0x7fa68819bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 --2- 172.21.15.112:0/1101750240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa688107cc0 0x7fa68819c0f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:20.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.815+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 >> 172.21.15.112:0/1101750240 conn(0x7fa6880fec90 msgr2=0x7fa688109570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:20.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.816+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 shutdown_connections 2024-04-25T16:37:20.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:20.816+0000 7fa68e05a700 1 -- 172.21.15.112:0/1101750240 wait complete. 2024-04-25T16:37:21.393 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:21 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1101750240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T16:37:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:21 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1101750240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T16:37:21.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:21 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1101750240' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T16:37:22.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:22 smithi112 ceph-mon[29122]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T16:37:22.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:22 smithi116 ceph-mon[32712]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T16:37:22.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:22 smithi133 ceph-mon[32691]: pgmap v310: 97 pgs: 1 peering, 96 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T16:37:23.118 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710740 got 300647710744 for osd.11 2024-04-25T16:37:23.118 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:23.306 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:23.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.337+0000 7ff630780700 1 -- 172.21.15.112:0/510871879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628103390 msgr2=0x7ff6281037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:23.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.337+0000 7ff630780700 1 --2- 172.21.15.112:0/510871879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628103390 0x7ff6281037a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff61800b0d0 tx=0x7ff61800b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 -- 172.21.15.112:0/510871879 shutdown_connections 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 --2- 172.21.15.112:0/510871879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff628108610 0x7ff62810ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 --2- 172.21.15.112:0/510871879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628103390 0x7ff6281037a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 --2- 172.21.15.112:0/510871879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff628107cc0 0x7ff6281080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 -- 172.21.15.112:0/510871879 >> 172.21.15.112:0/510871879 conn(0x7ff6280fec90 msgr2=0x7ff6281010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 -- 172.21.15.112:0/510871879 shutdown_connections 2024-04-25T16:37:23.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 -- 172.21.15.112:0/510871879 wait complete. 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 Processor -- start 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.338+0000 7ff630780700 1 -- start start 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff630780700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff628103390 0x7ff62819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff630780700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff628107cc0 0x7ff62819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff630780700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 0x7ff62819c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff630780700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff628113240 con 0x7ff628108610 2024-04-25T16:37:23.339 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff630780700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281130c0 con 0x7ff628107cc0 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff630780700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281133c0 con 0x7ff628103390 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 0x7ff62819c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 0x7ff62819c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55990/0 (socket says 172.21.15.112:55990) 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 -- 172.21.15.112:0/426372361 learned_addr learned my addr 172.21.15.112:0/426372361 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62dd1b700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff628107cc0 0x7ff62819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62e51c700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff628103390 0x7ff62819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 -- 172.21.15.112:0/426372361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff628103390 msgr2=0x7ff62819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff628103390 0x7ff62819bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 -- 172.21.15.112:0/426372361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff628107cc0 msgr2=0x7ff62819c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff628107cc0 0x7ff62819c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 -- 172.21.15.112:0/426372361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281a6e80 con 0x7ff628108610 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.339+0000 7ff62ed1d700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 0x7ff62819c660 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff620007ed0 tx=0x7ff62000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:23.340 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.340+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620012c90 con 0x7ff628108610 2024-04-25T16:37:23.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.340+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff620012e30 con 0x7ff628108610 2024-04-25T16:37:23.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.340+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620005280 con 0x7ff628108610 2024-04-25T16:37:23.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.340+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff628199a10 con 0x7ff628108610 2024-04-25T16:37:23.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.340+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff628199f00 con 0x7ff628108610 2024-04-25T16:37:23.341 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.341+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff628051190 con 0x7ff628108610 2024-04-25T16:37:23.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.342+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7ff620004750 con 0x7ff628108610 2024-04-25T16:37:23.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.343+0000 7ff6277fe700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6100afc10 0x7ff6100b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:23.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.343+0000 7ff62e51c700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6100afc10 0x7ff6100b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:23.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.343+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff6200e2ab0 con 0x7ff628108610 2024-04-25T16:37:23.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.344+0000 7ff62e51c700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6100afc10 0x7ff6100b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff618000f80 tx=0x7ff61800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:23.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.347+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6200ab620 con 0x7ff628108610 2024-04-25T16:37:23.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.554+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff62802e510 con 0x7ff628108610 2024-04-25T16:37:23.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.555+0000 7ff6277fe700 1 -- 172.21.15.112:0/426372361 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff6200aaf20 con 0x7ff628108610 2024-04-25T16:37:23.553 INFO:teuthology.orchestra.run.smithi112.stdout:253403070500 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6100afc10 msgr2=0x7ff6100b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6100afc10 0x7ff6100b2120 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff618000f80 tx=0x7ff61800b040 comp rx=0 tx=0).stop 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 msgr2=0x7ff62819c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 0x7ff62819c660 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff620007ed0 tx=0x7ff62000dbb0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 shutdown_connections 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff628108610 0x7ff62819c660 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff628103390 0x7ff62819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff6100afc10 0x7ff6100b2120 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 --2- 172.21.15.112:0/426372361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff628107cc0 0x7ff62819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:23.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 >> 172.21.15.112:0/426372361 conn(0x7ff6280fec90 msgr2=0x7ff628101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 shutdown_connections 2024-04-25T16:37:23.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:23.557+0000 7ff630780700 1 -- 172.21.15.112:0/426372361 wait complete. 2024-04-25T16:37:24.325 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070495 got 253403070500 for osd.9 2024-04-25T16:37:24.325 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:24.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:24 smithi112 ceph-mon[29122]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:24.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:24 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/426372361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T16:37:24.501 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:24 smithi133 ceph-mon[32691]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:24.501 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:24 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/426372361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T16:37:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:24 smithi116 ceph-mon[32712]: pgmap v311: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:24.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:24 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/426372361' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T16:37:24.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 -- 172.21.15.112:0/3205129679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610104150 msgr2=0x7fc61010ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:24.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 --2- 172.21.15.112:0/3205129679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610104150 0x7fc61010ff10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc60800b0a0 tx=0x7fc60800b3b0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 -- 172.21.15.112:0/3205129679 shutdown_connections 2024-04-25T16:37:24.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 --2- 172.21.15.112:0/3205129679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610104150 0x7fc61010ff10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 --2- 172.21.15.112:0/3205129679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6101080a0 0x7fc6101084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 --2- 172.21.15.112:0/3205129679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc610103800 0x7fc610103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 -- 172.21.15.112:0/3205129679 >> 172.21.15.112:0/3205129679 conn(0x7fc6100fe3f0 msgr2=0x7fc610100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 -- 172.21.15.112:0/3205129679 shutdown_connections 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.527+0000 7fc616897700 1 -- 172.21.15.112:0/3205129679 wait complete. 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 Processor -- start 2024-04-25T16:37:24.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 -- start start 2024-04-25T16:37:24.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610103800 0x7fc61019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:24.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc610104150 0x7fc61019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:24.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6101080a0 0x7fc610199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:24.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc610113230 con 0x7fc610103800 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101130b0 con 0x7fc6101080a0 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc616897700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6101133b0 con 0x7fc610104150 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610103800 0x7fc61019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc60ffff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610103800 0x7fc61019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56000/0 (socket says 172.21.15.112:56000) 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.528+0000 7fc60ffff700 1 -- 172.21.15.112:0/3196360129 learned_addr learned my addr 172.21.15.112:0/3196360129 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc610104150 0x7fc61019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc614e34700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6101080a0 0x7fc610199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 -- 172.21.15.112:0/3196360129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6101080a0 msgr2=0x7fc610199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6101080a0 0x7fc610199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 -- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610103800 msgr2=0x7fc61019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610103800 0x7fc61019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 -- 172.21.15.112:0/3196360129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61019a020 con 0x7fc610104150 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60f7fe700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc610104150 0x7fc61019fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc60400e9f0 tx=0x7fc60400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:24.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60400c9c0 con 0x7fc610104150 2024-04-25T16:37:24.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc60400cb60 con 0x7fc610104150 2024-04-25T16:37:24.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61019a310 con 0x7fc610104150 2024-04-25T16:37:24.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604018680 con 0x7fc610104150 2024-04-25T16:37:24.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.529+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6101a65e0 con 0x7fc610104150 2024-04-25T16:37:24.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.530+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc610051190 con 0x7fc610104150 2024-04-25T16:37:24.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.533+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fc604010430 con 0x7fc610104150 2024-04-25T16:37:24.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.533+0000 7fc60d7fa700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc5f80afc90 0x7fc5f80b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:24.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.533+0000 7fc60ffff700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc5f80afc90 0x7fc5f80b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:24.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.534+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc6040df2f0 con 0x7fc610104150 2024-04-25T16:37:24.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.534+0000 7fc60ffff700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc5f80afc90 0x7fc5f80b21a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc61019aab0 tx=0x7fc600009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:24.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.536+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6040a7d50 con 0x7fc610104150 2024-04-25T16:37:24.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.768+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc61002e510 con 0x7fc610104150 2024-04-25T16:37:24.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.768+0000 7fc60d7fa700 1 -- 172.21.15.112:0/3196360129 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc6040a7650 con 0x7fc610104150 2024-04-25T16:37:24.767 INFO:teuthology.orchestra.run.smithi112.stdout:107374182466 2024-04-25T16:37:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.770+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc5f80afc90 msgr2=0x7fc5f80b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.770+0000 7fc616897700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc5f80afc90 0x7fc5f80b21a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc61019aab0 tx=0x7fc600009260 comp rx=0 tx=0).stop 2024-04-25T16:37:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.770+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc610104150 msgr2=0x7fc61019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.770+0000 7fc616897700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc610104150 0x7fc61019fc90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc60400e9f0 tx=0x7fc60400ed00 comp rx=0 tx=0).stop 2024-04-25T16:37:24.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 shutdown_connections 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc5f80afc90 0x7fc5f80b21a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6101080a0 0x7fc610199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc610103800 0x7fc61019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 --2- 172.21.15.112:0/3196360129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc610104150 0x7fc61019fc90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 >> 172.21.15.112:0/3196360129 conn(0x7fc6100fe3f0 msgr2=0x7fc610109530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 shutdown_connections 2024-04-25T16:37:24.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:24.771+0000 7fc616897700 1 -- 172.21.15.112:0/3196360129 wait complete. 2024-04-25T16:37:25.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:25 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:37:25.359 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:25 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3196360129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T16:37:25.457 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182466 for osd.3 2024-04-25T16:37:25.457 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:25.487 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:25 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:37:25.488 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:25 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3196360129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T16:37:25.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:25 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:37:25.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:25 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3196360129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T16:37:26.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 -- 172.21.15.112:0/4208691770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105330 msgr2=0x7f8aa8105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:26.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 --2- 172.21.15.112:0/4208691770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105330 0x7f8aa8105740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009990 tx=0x7f8a9c009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 -- 172.21.15.112:0/4208691770 shutdown_connections 2024-04-25T16:37:26.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 --2- 172.21.15.112:0/4208691770 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 0x7f8aa8110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 --2- 172.21.15.112:0/4208691770 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105330 0x7f8aa8105740 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 --2- 172.21.15.112:0/4208691770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105c80 0x7f8aa810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 -- 172.21.15.112:0/4208691770 >> 172.21.15.112:0/4208691770 conn(0x7f8aa80fec90 msgr2=0x7f8aa81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:26.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 -- 172.21.15.112:0/4208691770 shutdown_connections 2024-04-25T16:37:26.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.044+0000 7f8aadd27700 1 -- 172.21.15.112:0/4208691770 wait complete. 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.045+0000 7f8aadd27700 1 Processor -- start 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.045+0000 7f8aadd27700 1 -- start start 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.045+0000 7f8aadd27700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105330 0x7f8aa81a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aadd27700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105c80 0x7f8aa81a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aadd27700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 0x7f8aa819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aadd27700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81142f0 con 0x7f8aa8105c80 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aadd27700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8114170 con 0x7f8aa810e7a0 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aadd27700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8114470 con 0x7f8aa8105330 2024-04-25T16:37:26.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105c80 0x7f8aa81a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 0x7f8aa819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105330 0x7f8aa81a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105c80 0x7f8aa81a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52594/0 (socket says 172.21.15.112:52594) 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 0x7f8aa819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:34866/0 (socket says 172.21.15.112:34866) 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa6ffd700 1 -- 172.21.15.112:0/3486694635 learned_addr learned my addr 172.21.15.112:0/3486694635 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 -- 172.21.15.112:0/3486694635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 msgr2=0x7f8aa819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 0x7f8aa819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 -- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105c80 msgr2=0x7f8aa81a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105c80 0x7f8aa81a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 -- 172.21.15.112:0/3486694635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa819b160 con 0x7f8aa8105330 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa77fe700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105330 0x7f8aa81a0890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0059d0 tx=0x7f8a9c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c027070 con 0x7f8aa8105330 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa819b3f0 con 0x7f8aa8105330 2024-04-25T16:37:26.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a9c022470 con 0x7f8aa8105330 2024-04-25T16:37:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.046+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c017650 con 0x7f8aa8105330 2024-04-25T16:37:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.047+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa819b8f0 con 0x7f8aa8105330 2024-04-25T16:37:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.047+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa8051190 con 0x7f8aa8105330 2024-04-25T16:37:26.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.049+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f8a9c02a070 con 0x7f8aa8105330 2024-04-25T16:37:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.049+0000 7f8aa4ff9700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a900afc10 0x7f8a900b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.050+0000 7f8aa6ffd700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a900afc10 0x7f8a900b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.050+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8a9c0dfef0 con 0x7f8aa8105330 2024-04-25T16:37:26.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.050+0000 7f8aa6ffd700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a900afc10 0x7f8a900b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8102180 tx=0x7f8a98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:26.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.053+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a9c013070 con 0x7f8aa8105330 2024-04-25T16:37:26.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.273+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8aa802e510 con 0x7f8aa8105330 2024-04-25T16:37:26.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.274+0000 7f8aa4ff9700 1 -- 172.21.15.112:0/3486694635 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a9c0a89e0 con 0x7f8aa8105330 2024-04-25T16:37:26.273 INFO:teuthology.orchestra.run.smithi112.stdout:201863462958 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a900afc10 msgr2=0x7f8a900b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a900afc10 0x7f8a900b2120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8102180 tx=0x7f8a98009260 comp rx=0 tx=0).stop 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105330 msgr2=0x7f8aa81a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105330 0x7f8aa81a0890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0059d0 tx=0x7f8a9c0047b0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 shutdown_connections 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a900afc10 0x7f8a900b2120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8aa810e7a0 0x7f8aa819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8aa8105330 0x7f8aa81a0890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 --2- 172.21.15.112:0/3486694635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8aa8105c80 0x7f8aa81a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:26.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 >> 172.21.15.112:0/3486694635 conn(0x7f8aa80fec90 msgr2=0x7f8aa80ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:26.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 shutdown_connections 2024-04-25T16:37:26.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:26.276+0000 7f8aadd27700 1 -- 172.21.15.112:0/3486694635 wait complete. 2024-04-25T16:37:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:26 smithi116 ceph-mon[32712]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:26 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.529 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:26 smithi112 ceph-mon[29122]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:26.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.530 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:26 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:26 smithi133 ceph-mon[32691]: pgmap v312: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:26.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:26.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:26 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:27.103 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:27.504 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:27 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3486694635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T16:37:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:27 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3486694635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T16:37:27.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:27 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3486694635' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T16:37:28.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:28 smithi116 ceph-mon[32712]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:28.532 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:28 smithi112 ceph-mon[29122]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 -- 172.21.15.112:0/1016268672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701091a0 msgr2=0x7f90701095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 --2- 172.21.15.112:0/1016268672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701091a0 0x7f90701095b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9060009990 tx=0x7f9060009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 -- 172.21.15.112:0/1016268672 shutdown_connections 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 --2- 172.21.15.112:0/1016268672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102f80 0x7f9070111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 --2- 172.21.15.112:0/1016268672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90701091a0 0x7f90701095b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 --2- 172.21.15.112:0/1016268672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9070102630 0x7f9070102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 -- 172.21.15.112:0/1016268672 >> 172.21.15.112:0/1016268672 conn(0x7f90700fe3f0 msgr2=0x7f9070100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:28.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 -- 172.21.15.112:0/1016268672 shutdown_connections 2024-04-25T16:37:28.649 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.649+0000 7f9075940700 1 -- 172.21.15.112:0/1016268672 wait complete. 2024-04-25T16:37:28.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.650+0000 7f9075940700 1 Processor -- start 2024-04-25T16:37:28.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.650+0000 7f9075940700 1 -- start start 2024-04-25T16:37:28.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f9075940700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102630 0x7f907019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:28.650 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f9075940700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 0x7f907019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f9075940700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90701091a0 0x7f907019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f9075940700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9070114430 con 0x7f9070102f80 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f9075940700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90701142b0 con 0x7f90701091a0 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f9075940700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90701145b0 con 0x7f9070102630 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 0x7f907019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 0x7f907019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52640/0 (socket says 172.21.15.112:52640) 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 -- 172.21.15.112:0/3838037165 learned_addr learned my addr 172.21.15.112:0/3838037165 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906f7fe700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90701091a0 0x7f907019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906effd700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102630 0x7f907019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 -- 172.21.15.112:0/3838037165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102630 msgr2=0x7f907019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102630 0x7f907019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 -- 172.21.15.112:0/3838037165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90701091a0 msgr2=0x7f907019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90701091a0 0x7f907019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 -- 172.21.15.112:0/3838037165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907019fcb0 con 0x7f9070102f80 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906effd700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102630 0x7f907019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.651+0000 7f906e7fc700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 0x7f907019b380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f906400e970 tx=0x7f906400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.652+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906400cbd0 con 0x7f9070102f80 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.652+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906400cd70 con 0x7f9070102f80 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.652+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907019ffa0 con 0x7f9070102f80 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.652+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9064005680 con 0x7f9070102f80 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.652+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90701a78c0 con 0x7f9070102f80 2024-04-25T16:37:28.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.653+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9070051190 con 0x7f9070102f80 2024-04-25T16:37:28.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.654+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f9064004750 con 0x7f9070102f80 2024-04-25T16:37:28.653 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.655+0000 7f907493e700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f90580afc10 0x7f90580b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:28.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.656+0000 7f906effd700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f90580afc10 0x7f90580b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:28.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.656+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f90640df290 con 0x7f9070102f80 2024-04-25T16:37:28.654 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.656+0000 7f906effd700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f90580afc10 0x7f90580b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9060006010 tx=0x7f906000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:28.658 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.659+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9064053ff0 con 0x7f9070102f80 2024-04-25T16:37:28.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:28 smithi133 ceph-mon[32691]: pgmap v313: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T16:37:28.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.880+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f907019c140 con 0x7f9070102f80 2024-04-25T16:37:28.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.880+0000 7f907493e700 1 -- 172.21.15.112:0/3838037165 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f90640a7c90 con 0x7f9070102f80 2024-04-25T16:37:28.879 INFO:teuthology.orchestra.run.smithi112.stdout:274877906975 2024-04-25T16:37:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f90580afc10 msgr2=0x7f90580b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f90580afc10 0x7f90580b2120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9060006010 tx=0x7f906000b260 comp rx=0 tx=0).stop 2024-04-25T16:37:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 msgr2=0x7f907019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 0x7f907019b380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f906400e970 tx=0x7f906400ec80 comp rx=0 tx=0).stop 2024-04-25T16:37:28.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 shutdown_connections 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f90580afc10 0x7f90580b2120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90701091a0 0x7f907019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9070102630 0x7f907019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 --2- 172.21.15.112:0/3838037165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9070102f80 0x7f907019b380 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 >> 172.21.15.112:0/3838037165 conn(0x7f90700fe3f0 msgr2=0x7f9070100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 shutdown_connections 2024-04-25T16:37:28.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.883+0000 7f9075940700 1 -- 172.21.15.112:0/3838037165 wait complete. 2024-04-25T16:37:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 -- 172.21.15.112:0/2806362215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed81091a0 msgr2=0x7efed81095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 --2- 172.21.15.112:0/2806362215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed81091a0 0x7efed81095b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efec8009990 tx=0x7efec8009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 -- 172.21.15.112:0/2806362215 shutdown_connections 2024-04-25T16:37:29.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 --2- 172.21.15.112:0/2806362215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed8102f80 0x7efed8111010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 --2- 172.21.15.112:0/2806362215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed81091a0 0x7efed81095b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 --2- 172.21.15.112:0/2806362215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102630 0x7efed8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 -- 172.21.15.112:0/2806362215 >> 172.21.15.112:0/2806362215 conn(0x7efed80fe3f0 msgr2=0x7efed8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 -- 172.21.15.112:0/2806362215 shutdown_connections 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.554+0000 7efede34c700 1 -- 172.21.15.112:0/2806362215 wait complete. 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 Processor -- start 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 -- start start 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 0x7efed8103810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102f80 0x7efed8103d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed81091a0 0x7efed8104290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efed8114510 con 0x7efed8102630 2024-04-25T16:37:29.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efed8114390 con 0x7efed8102f80 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.555+0000 7efede34c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efed8114690 con 0x7efed81091a0 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 0x7efed8103810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 0x7efed8103810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52612/0 (socket says 172.21.15.112:52612) 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 -- 172.21.15.112:0/2849629800 learned_addr learned my addr 172.21.15.112:0/2849629800 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed77fe700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102f80 0x7efed8103d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efedc8e9700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed81091a0 0x7efed8104290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 -- 172.21.15.112:0/2849629800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed81091a0 msgr2=0x7efed8104290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed81091a0 0x7efed8104290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 -- 172.21.15.112:0/2849629800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102f80 msgr2=0x7efed8103d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102f80 0x7efed8103d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 -- 172.21.15.112:0/2849629800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed81a9240 con 0x7efed8102630 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed7fff700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 0x7efed8103810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efec8006010 tx=0x7efec80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:29.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8027070 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed81a94d0 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efec8004020 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8022db0 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.556+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81a9980 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.557+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed810c740 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.560+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7efec802a070 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.560+0000 7efed57fa700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efec00afc10 0x7efec00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.561+0000 7efed77fe700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efec00afc10 0x7efec00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.561+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7efec80e04a0 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.561+0000 7efed77fe700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efec00afc10 0x7efec00b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efed8107940 tx=0x7efecc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.563+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efec8013070 con 0x7efed8102630 2024-04-25T16:37:29.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.773+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7efed8104d40 con 0x7efed8102630 2024-04-25T16:37:29.375 INFO:teuthology.orchestra.run.smithi112.stdout:227633266729 2024-04-25T16:37:29.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.774+0000 7efed57fa700 1 -- 172.21.15.112:0/2849629800 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efec80a8f30 con 0x7efed8102630 2024-04-25T16:37:29.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efec00afc10 msgr2=0x7efec00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efec00afc10 0x7efec00b2120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efed8107940 tx=0x7efecc009190 comp rx=0 tx=0).stop 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 msgr2=0x7efed8103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 0x7efed8103810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efec8006010 tx=0x7efec80047b0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 shutdown_connections 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efed81091a0 0x7efed8104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efed8102630 0x7efed8103810 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efec00afc10 0x7efec00b2120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 --2- 172.21.15.112:0/2849629800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efed8102f80 0x7efed8103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 >> 172.21.15.112:0/2849629800 conn(0x7efed80fe3f0 msgr2=0x7efed8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 shutdown_connections 2024-04-25T16:37:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:28.776+0000 7efede34c700 1 -- 172.21.15.112:0/2849629800 wait complete. 2024-04-25T16:37:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2849629800' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T16:37:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3838037165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T16:37:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2849629800' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T16:37:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3838037165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T16:37:29.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2849629800' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T16:37:29.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3838037165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T16:37:30.196 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266725 got 227633266729 for osd.8 2024-04-25T16:37:30.196 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:30.419 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:30 smithi112 ceph-mon[29122]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:30.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 -- 172.21.15.112:0/32137532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c1085d0 msgr2=0x7f8a4c10ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:30.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 --2- 172.21.15.112:0/32137532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c10ff30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400b0d0 tx=0x7f8a4400b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 -- 172.21.15.112:0/32137532 shutdown_connections 2024-04-25T16:37:30.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 --2- 172.21.15.112:0/32137532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c10ff30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 --2- 172.21.15.112:0/32137532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4c103350 0x7f8a4c103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 --2- 172.21.15.112:0/32137532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c107c80 0x7f8a4c108090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 -- 172.21.15.112:0/32137532 >> 172.21.15.112:0/32137532 conn(0x7f8a4c0fec30 msgr2=0x7f8a4c101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:30.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 -- 172.21.15.112:0/32137532 shutdown_connections 2024-04-25T16:37:30.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.425+0000 7f8a51108700 1 -- 172.21.15.112:0/32137532 wait complete. 2024-04-25T16:37:30.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.426+0000 7f8a51108700 1 Processor -- start 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.426+0000 7f8a51108700 1 -- start start 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a51108700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 0x7f8a4c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a51108700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4c107c80 0x7f8a4c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a51108700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a51108700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c113200 con 0x7f8a4c103350 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a51108700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c113080 con 0x7f8a4c1085d0 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a51108700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4c113380 con 0x7f8a4c107c80 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 0x7f8a4c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 0x7f8a4c19ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52656/0 (socket says 172.21.15.112:52656) 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 -- 172.21.15.112:0/1464892802 learned_addr learned my addr 172.21.15.112:0/1464892802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:30.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4a59c700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4c107c80 0x7f8a4c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4b59e700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 -- 172.21.15.112:0/1464892802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4c107c80 msgr2=0x7f8a4c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4c107c80 0x7f8a4c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 -- 172.21.15.112:0/1464892802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c1085d0 msgr2=0x7f8a4c19c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c19c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 -- 172.21.15.112:0/1464892802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4c1a6df0 con 0x7f8a4c103350 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4b59e700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a4ad9d700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 0x7f8a4c19ba80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d950 tx=0x7f8a3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.427+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c016070 con 0x7f8a4c103350 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.428+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4c1a70e0 con 0x7f8a4c103350 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.428+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a3c009df0 con 0x7f8a4c103350 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.428+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c00f3c0 con 0x7f8a4c103350 2024-04-25T16:37:30.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.428+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4c1a7670 con 0x7f8a4c103350 2024-04-25T16:37:30.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.429+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a4c051190 con 0x7f8a4c103350 2024-04-25T16:37:30.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.430+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f8a3c019070 con 0x7f8a4c103350 2024-04-25T16:37:30.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.430+0000 7f8a33fff700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a340afc10 0x7f8a340b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:30.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.431+0000 7f8a4a59c700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a340afc10 0x7f8a340b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:30.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.431+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8a3c0de9f0 con 0x7f8a4c103350 2024-04-25T16:37:30.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.431+0000 7f8a4a59c700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a340afc10 0x7f8a340b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c1065e0 tx=0x7f8a40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:30.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.435+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a3c0a7450 con 0x7f8a4c103350 2024-04-25T16:37:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:30 smithi116 ceph-mon[32712]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:30.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.640+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f8a4c02e510 con 0x7f8a4c103350 2024-04-25T16:37:30.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.640+0000 7f8a33fff700 1 -- 172.21.15.112:0/1464892802 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a3c0a6d50 con 0x7f8a4c103350 2024-04-25T16:37:30.639 INFO:teuthology.orchestra.run.smithi112.stdout:154618822714 2024-04-25T16:37:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a340afc10 msgr2=0x7f8a340b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a340afc10 0x7f8a340b2120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c1065e0 tx=0x7f8a40009260 comp rx=0 tx=0).stop 2024-04-25T16:37:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 msgr2=0x7f8a4c19ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 0x7f8a4c19ba80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d950 tx=0x7f8a3c00dc60 comp rx=0 tx=0).stop 2024-04-25T16:37:30.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 shutdown_connections 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8a340afc10 0x7f8a340b2120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a4c1085d0 0x7f8a4c19c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8a4c103350 0x7f8a4c19ba80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 --2- 172.21.15.112:0/1464892802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a4c107c80 0x7f8a4c19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 >> 172.21.15.112:0/1464892802 conn(0x7f8a4c0fec30 msgr2=0x7f8a4c1073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 shutdown_connections 2024-04-25T16:37:30.642 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:30.642+0000 7f8a51108700 1 -- 172.21.15.112:0/1464892802 wait complete. 2024-04-25T16:37:30.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:30 smithi133 ceph-mon[32691]: pgmap v314: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:31.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462953 got 201863462958 for osd.7 2024-04-25T16:37:31.116 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:31.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:31 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1464892802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T16:37:31.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:31 smithi112 ceph-mon[29122]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:31 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1464892802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T16:37:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:31 smithi116 ceph-mon[32712]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:31 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1464892802' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T16:37:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:31 smithi133 ceph-mon[32691]: pgmap v315: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:31.943 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906970 got 274877906975 for osd.10 2024-04-25T16:37:31.944 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:32.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822710 got 154618822714 for osd.5 2024-04-25T16:37:32.998 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:33.901 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:34.276 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:34 smithi112 ceph-mon[29122]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:34.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:34 smithi133 ceph-mon[32691]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:34 smithi116 ceph-mon[32712]: pgmap v316: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 -- 172.21.15.112:0/2588576159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c1096f0 msgr2=0x7fa51c111050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 --2- 172.21.15.112:0/2588576159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c1096f0 0x7fa51c111050 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa51400b0d0 tx=0x7fa51400b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 -- 172.21.15.112:0/2588576159 shutdown_connections 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 --2- 172.21.15.112:0/2588576159 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c1096f0 0x7fa51c111050 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 --2- 172.21.15.112:0/2588576159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa51c102980 0x7fa51c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 --2- 172.21.15.112:0/2588576159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa51c108da0 0x7fa51c1091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 -- 172.21.15.112:0/2588576159 >> 172.21.15.112:0/2588576159 conn(0x7fa51c0fe3f0 msgr2=0x7fa51c100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 -- 172.21.15.112:0/2588576159 shutdown_connections 2024-04-25T16:37:34.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.881+0000 7fa523b31700 1 -- 172.21.15.112:0/2588576159 wait complete. 2024-04-25T16:37:34.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 Processor -- start 2024-04-25T16:37:34.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 -- start start 2024-04-25T16:37:34.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 0x7fa51c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa51c108da0 0x7fa51c1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa51c1096f0 0x7fa51c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c114320 con 0x7fa51c102980 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c1141a0 con 0x7fa51c108da0 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa523b31700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c1144a0 con 0x7fa51c1096f0 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa5218cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 0x7fa51c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa5218cd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 0x7fa51c1a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52682/0 (socket says 172.21.15.112:52682) 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.882+0000 7fa5218cd700 1 -- 172.21.15.112:0/2392177885 learned_addr learned my addr 172.21.15.112:0/2392177885 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5210cc700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa51c108da0 0x7fa51c1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5220ce700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa51c1096f0 0x7fa51c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5218cd700 1 -- 172.21.15.112:0/2392177885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa51c1096f0 msgr2=0x7fa51c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5218cd700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa51c1096f0 0x7fa51c19aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5218cd700 1 -- 172.21.15.112:0/2392177885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa51c108da0 msgr2=0x7fa51c1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5218cd700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa51c108da0 0x7fa51c1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:34.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5218cd700 1 -- 172.21.15.112:0/2392177885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c19b0a0 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa5218cd700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 0x7fa51c1a0840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c9b0 tx=0x7fa50c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c007b00 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa50c007ca0 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51c19b390 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c018730 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.883+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51c1a76d0 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.884+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51c051190 con 0x7fa51c102980 2024-04-25T16:37:34.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.886+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fa50c00f450 con 0x7fa51c102980 2024-04-25T16:37:34.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.886+0000 7fa512ffd700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5080afc90 0x7fa5080b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:34.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.887+0000 7fa5210cc700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5080afc90 0x7fa5080b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:34.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.887+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa50c0df0e0 con 0x7fa51c102980 2024-04-25T16:37:34.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.887+0000 7fa5210cc700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5080afc90 0x7fa5080b21a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa518005fd0 tx=0x7fa518005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:34.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:34.890+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa50c014030 con 0x7fa51c102980 2024-04-25T16:37:35.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.100+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa51c19bfc0 con 0x7fa51c102980 2024-04-25T16:37:35.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.100+0000 7fa512ffd700 1 -- 172.21.15.112:0/2392177885 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa51c19bfc0 con 0x7fa51c102980 2024-04-25T16:37:35.709 INFO:teuthology.orchestra.run.smithi112.stdout:77309411401 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5080afc90 msgr2=0x7fa5080b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5080afc90 0x7fa5080b21a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa518005fd0 tx=0x7fa518005db0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 msgr2=0x7fa51c1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 0x7fa51c1a0840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c9b0 tx=0x7fa50c00cd70 comp rx=0 tx=0).stop 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 shutdown_connections 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa51c1096f0 0x7fa51c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa51c102980 0x7fa51c1a0840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5080afc90 0x7fa5080b21a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 --2- 172.21.15.112:0/2392177885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa51c108da0 0x7fa51c1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 >> 172.21.15.112:0/2392177885 conn(0x7fa51c0fe3f0 msgr2=0x7fa51c10a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:35.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 shutdown_connections 2024-04-25T16:37:35.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.102+0000 7fa523b31700 1 -- 172.21.15.112:0/2392177885 wait complete. 2024-04-25T16:37:35.711 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:35 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2392177885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T16:37:35.712 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:35 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2392177885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T16:37:35.712 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:35 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2392177885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 -- 172.21.15.112:0/1055280851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c105350 msgr2=0x7f821c105760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 --2- 172.21.15.112:0/1055280851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c105350 0x7f821c105760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8218009990 tx=0x7f8218009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 -- 172.21.15.112:0/1055280851 shutdown_connections 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 --2- 172.21.15.112:0/1055280851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c10e780 0x7f821c110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 --2- 172.21.15.112:0/1055280851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c105350 0x7f821c105760 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 --2- 172.21.15.112:0/1055280851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f821c105ca0 0x7f821c10e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 -- 172.21.15.112:0/1055280851 >> 172.21.15.112:0/1055280851 conn(0x7f821c0fec30 msgr2=0x7f821c101050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 -- 172.21.15.112:0/1055280851 shutdown_connections 2024-04-25T16:37:35.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.932+0000 7f82238a1700 1 -- 172.21.15.112:0/1055280851 wait complete. 2024-04-25T16:37:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 Processor -- start 2024-04-25T16:37:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 -- start start 2024-04-25T16:37:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f821c105350 0x7f821c1a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c105ca0 0x7f821c1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 0x7f821c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:35.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f821c114350 con 0x7f821c10e780 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1141d0 con 0x7f821c105350 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f82238a1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1144d0 con 0x7f821c105ca0 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 0x7f821c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 0x7f821c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49444/0 (socket says 172.21.15.112:49444) 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 -- 172.21.15.112:0/2184842963 learned_addr learned my addr 172.21.15.112:0/2184842963 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f822163d700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f821c105350 0x7f821c1a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8220e3c700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c105ca0 0x7f821c1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 -- 172.21.15.112:0/2184842963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c105ca0 msgr2=0x7f821c1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c105ca0 0x7f821c1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 -- 172.21.15.112:0/2184842963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f821c105350 msgr2=0x7f821c1a0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f821c105350 0x7f821c1a0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.933+0000 7f8221e3e700 1 -- 172.21.15.112:0/2184842963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c19b130 con 0x7f821c10e780 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.934+0000 7f8221e3e700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 0x7f821c19aaf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f821400b970 tx=0x7f821400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:35.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.934+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821400d860 con 0x7f821c10e780 2024-04-25T16:37:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.934+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c19b420 con 0x7f821c10e780 2024-04-25T16:37:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.934+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821400e430 con 0x7f821c10e780 2024-04-25T16:37:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.934+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214017660 con 0x7f821c10e780 2024-04-25T16:37:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.934+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c19b910 con 0x7f821c10e780 2024-04-25T16:37:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.935+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f821c051190 con 0x7f821c10e780 2024-04-25T16:37:35.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.936+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f821400da00 con 0x7f821c10e780 2024-04-25T16:37:35.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.937+0000 7f82127fc700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82080afc10 0x7f82080b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:35.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.937+0000 7f822163d700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82080afc10 0x7f82080b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:35.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.937+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f82140debf0 con 0x7f821c10e780 2024-04-25T16:37:35.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.937+0000 7f822163d700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82080afc10 0x7f82080b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8218009750 tx=0x7f821801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:35.939 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:35.941+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8214052ff0 con 0x7f821c10e780 2024-04-25T16:37:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.149+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f821c02e510 con 0x7f821c10e780 2024-04-25T16:37:36.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.149+0000 7f82127fc700 1 -- 172.21.15.112:0/2184842963 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8214013030 con 0x7f821c10e780 2024-04-25T16:37:36.148 INFO:teuthology.orchestra.run.smithi112.stdout:55834574927 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82080afc10 msgr2=0x7f82080b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82080afc10 0x7f82080b2120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8218009750 tx=0x7f821801a040 comp rx=0 tx=0).stop 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 msgr2=0x7f821c19aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 0x7f821c19aaf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f821400b970 tx=0x7f821400bc80 comp rx=0 tx=0).stop 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 shutdown_connections 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f821c10e780 0x7f821c19aaf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f82080afc10 0x7f82080b2120 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f821c105350 0x7f821c1a0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:36.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 --2- 172.21.15.112:0/2184842963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821c105ca0 0x7f821c1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 >> 172.21.15.112:0/2184842963 conn(0x7f821c0fec30 msgr2=0x7f821c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 shutdown_connections 2024-04-25T16:37:36.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:36.151+0000 7f82238a1700 1 -- 172.21.15.112:0/2184842963 wait complete. 2024-04-25T16:37:36.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:36 smithi133 ceph-mon[32691]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:37:36.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:36 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2184842963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T16:37:36.481 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:36 smithi112 ceph-mon[29122]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:37:36.482 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:36 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2184842963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T16:37:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:36 smithi116 ceph-mon[32712]: pgmap v317: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:37:36.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:36 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2184842963' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T16:37:37.404 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574927 for osd.1 2024-04-25T16:37:37.405 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:38.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411401 for osd.2 2024-04-25T16:37:38.059 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:38.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:38 smithi112 ceph-mon[29122]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:37:38.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:38 smithi133 ceph-mon[32691]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:37:38.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:38 smithi116 ceph-mon[32712]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:37:39.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/3744887974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8108610 msgr2=0x7f4bc810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/3744887974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8108610 0x7f4bc810ff70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000b0d0 tx=0x7f4bc000b3e0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/3744887974 shutdown_connections 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/3744887974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8108610 0x7f4bc810ff70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/3744887974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc8103390 0x7f4bc81037a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/3744887974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bc8107cc0 0x7f4bc81080d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/3744887974 >> 172.21.15.112:0/3744887974 conn(0x7f4bc80fec90 msgr2=0x7f4bc81010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.014+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/3744887974 shutdown_connections 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.015+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/3744887974 wait complete. 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.015+0000 7f4bcd7b3700 1 Processor -- start 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.015+0000 7f4bcd7b3700 1 -- start start 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 0x7f4bc81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc8107cc0 0x7f4bc81a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bc8108610 0x7f4bc819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc8113240 con 0x7f4bc8103390 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc81130c0 con 0x7f4bc8107cc0 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc81133c0 con 0x7f4bc8108610 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 0x7f4bc81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 0x7f4bc81a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49454/0 (socket says 172.21.15.112:49454) 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 -- 172.21.15.112:0/335338008 learned_addr learned my addr 172.21.15.112:0/335338008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc67fc700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc8107cc0 0x7f4bc81a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc77fe700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bc8108610 0x7f4bc819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 -- 172.21.15.112:0/335338008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bc8108610 msgr2=0x7f4bc819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bc8108610 0x7f4bc819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 -- 172.21.15.112:0/335338008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc8107cc0 msgr2=0x7f4bc81a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc8107cc0 0x7f4bc81a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 -- 172.21.15.112:0/335338008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc819b120 con 0x7f4bc8103390 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bc6ffd700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 0x7f4bc81a0850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800d950 tx=0x7f4bb800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8016070 con 0x7f4bc8103390 2024-04-25T16:37:39.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc819b410 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bb8009df0 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800f3c0 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.016+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc81a76e0 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.017+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc8051190 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.019+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f4bb8019070 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.019+0000 7f4baffff700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4bb00afc10 0x7f4bb00b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.020+0000 7f4bc67fc700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4bb00afc10 0x7f4bb00b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.020+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f4bb80dea00 con 0x7f4bc8103390 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.020+0000 7f4bc67fc700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4bb00afc10 0x7f4bb00b2120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8109700 tx=0x7f4bbc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:39.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.023+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bb80a7460 con 0x7f4bc8103390 2024-04-25T16:37:39.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.227+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4bc802e510 con 0x7f4bc8103390 2024-04-25T16:37:39.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.227+0000 7f4baffff700 1 -- 172.21.15.112:0/335338008 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4bb80a6d60 con 0x7f4bc8103390 2024-04-25T16:37:39.226 INFO:teuthology.orchestra.run.smithi112.stdout:180388626486 2024-04-25T16:37:39.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4bb00afc10 msgr2=0x7f4bb00b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4bb00afc10 0x7f4bb00b2120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8109700 tx=0x7f4bbc0073d0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 msgr2=0x7f4bc81a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 0x7f4bc81a0850 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800d950 tx=0x7f4bb800dc60 comp rx=0 tx=0).stop 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 shutdown_connections 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4bc8108610 0x7f4bc819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bc8103390 0x7f4bc81a0850 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f4bb00afc10 0x7f4bb00b2120 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 --2- 172.21.15.112:0/335338008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bc8107cc0 0x7f4bc81a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 >> 172.21.15.112:0/335338008 conn(0x7f4bc80fec90 msgr2=0x7f4bc806b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 shutdown_connections 2024-04-25T16:37:39.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.230+0000 7f4bcd7b3700 1 -- 172.21.15.112:0/335338008 wait complete. 2024-04-25T16:37:39.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 -- 172.21.15.112:0/759891268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 msgr2=0x7f8ea41091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/759891268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea41091f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e980099c0 tx=0x7f8e98009cd0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 -- 172.21.15.112:0/759891268 shutdown_connections 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/759891268 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea4109730 0x7f8ea4111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/759891268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ea41044b0 0x7f8ea41048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/759891268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea41091f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 -- 172.21.15.112:0/759891268 >> 172.21.15.112:0/759891268 conn(0x7f8ea40ffda0 msgr2=0x7f8ea41021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 -- 172.21.15.112:0/759891268 shutdown_connections 2024-04-25T16:37:39.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 -- 172.21.15.112:0/759891268 wait complete. 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.668+0000 7f8ea9d15700 1 Processor -- start 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 -- start start 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ea41044b0 0x7f8ea419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea4109730 0x7f8ea419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea4114430 con 0x7f8ea4108de0 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41142b0 con 0x7f8ea4109730 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea9d15700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea41145b0 con 0x7f8ea41044b0 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49474/0 (socket says 172.21.15.112:49474) 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 -- 172.21.15.112:0/1218617845 learned_addr learned my addr 172.21.15.112:0/1218617845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:39.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea37fe700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ea41044b0 0x7f8ea419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 -- 172.21.15.112:0/1218617845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ea41044b0 msgr2=0x7f8ea419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ea41044b0 0x7f8ea419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 -- 172.21.15.112:0/1218617845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea4109730 msgr2=0x7f8ea419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea4109730 0x7f8ea419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 -- 172.21.15.112:0/1218617845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea419fc90 con 0x7f8ea4108de0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.669+0000 7f8ea2ffd700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea419b360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009f50 tx=0x7f8e980037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.670+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98027070 con 0x7f8ea4108de0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.670+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e98004090 con 0x7f8ea4108de0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.670+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea419ff20 con 0x7f8ea4108de0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.670+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98022e60 con 0x7f8ea4108de0 2024-04-25T16:37:39.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.670+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea41a0400 con 0x7f8ea4108de0 2024-04-25T16:37:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.671+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea4051190 con 0x7f8ea4108de0 2024-04-25T16:37:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.672+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f8e9802a070 con 0x7f8ea4108de0 2024-04-25T16:37:39.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.673+0000 7f8ea0ff9700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e8c0afc10 0x7f8e8c0b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:39.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.673+0000 7f8ea37fe700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e8c0afc10 0x7f8e8c0b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:39.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.673+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f8e980e00c0 con 0x7f8ea4108de0 2024-04-25T16:37:39.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.673+0000 7f8ea37fe700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e8c0afc10 0x7f8e8c0b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009a10 tx=0x7f8e94009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:39.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.677+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e98031080 con 0x7f8ea4108de0 2024-04-25T16:37:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.879+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f8ea402e510 con 0x7f8ea4108de0 2024-04-25T16:37:39.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.880+0000 7f8ea0ff9700 1 -- 172.21.15.112:0/1218617845 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e98013070 con 0x7f8ea4108de0 2024-04-25T16:37:39.879 INFO:teuthology.orchestra.run.smithi112.stdout:133143986241 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e8c0afc10 msgr2=0x7f8e8c0b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e8c0afc10 0x7f8e8c0b2120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009a10 tx=0x7f8e94009190 comp rx=0 tx=0).stop 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 msgr2=0x7f8ea419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea419b360 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009f50 tx=0x7f8e980037c0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 shutdown_connections 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e8c0afc10 0x7f8e8c0b2120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ea4109730 0x7f8ea419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ea41044b0 0x7f8ea419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 --2- 172.21.15.112:0/1218617845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8ea4108de0 0x7f8ea419b360 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 >> 172.21.15.112:0/1218617845 conn(0x7f8ea40ffda0 msgr2=0x7f8ea41085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 shutdown_connections 2024-04-25T16:37:39.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:39.882+0000 7f8ea9d15700 1 -- 172.21.15.112:0/1218617845 wait complete. 2024-04-25T16:37:40.118 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626479 got 180388626486 for osd.6 2024-04-25T16:37:40.118 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:40.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:40 smithi133 ceph-mon[32691]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T16:37:40.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:40 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/335338008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T16:37:40.452 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:40 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1218617845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T16:37:40.513 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:40 smithi116 ceph-mon[32712]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T16:37:40.513 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:40 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/335338008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T16:37:40.513 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:40 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1218617845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T16:37:40.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:40 smithi112 ceph-mon[29122]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T16:37:40.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:40 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/335338008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T16:37:40.600 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:40 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1218617845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 -- 172.21.15.112:0/3963735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102f80 msgr2=0x7f8190111010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 --2- 172.21.15.112:0/3963735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102f80 0x7f8190111010 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f818800b0a0 tx=0x7f818800b3b0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 -- 172.21.15.112:0/3963735075 shutdown_connections 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 --2- 172.21.15.112:0/3963735075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102f80 0x7f8190111010 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 --2- 172.21.15.112:0/3963735075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 0x7f81901095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 --2- 172.21.15.112:0/3963735075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102630 0x7f8190102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 -- 172.21.15.112:0/3963735075 >> 172.21.15.112:0/3963735075 conn(0x7f81900fe3f0 msgr2=0x7f8190100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 -- 172.21.15.112:0/3963735075 shutdown_connections 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.743+0000 7f81977c4700 1 -- 172.21.15.112:0/3963735075 wait complete. 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 Processor -- start 2024-04-25T16:37:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 -- start start 2024-04-25T16:37:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102630 0x7f819019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102f80 0x7f819019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 0x7f819019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8190114440 con 0x7f8190102630 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81901142c0 con 0x7f81901091a0 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.744+0000 7f81977c4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81901145c0 con 0x7f8190102f80 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195d61700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 0x7f819019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8194d5f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102f80 0x7f819019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195560700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102630 0x7f819019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8194d5f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102f80 0x7f819019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:37188/0 (socket says 172.21.15.112:37188) 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195d61700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 0x7f819019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:40354/0 (socket says 172.21.15.112:40354) 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8194d5f700 1 -- 172.21.15.112:0/229886756 learned_addr learned my addr 172.21.15.112:0/229886756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195560700 1 -- 172.21.15.112:0/229886756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102f80 msgr2=0x7f819019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195560700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102f80 0x7f819019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195560700 1 -- 172.21.15.112:0/229886756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 msgr2=0x7f819019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:40.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195560700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 0x7f819019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.745+0000 7f8195560700 1 -- 172.21.15.112:0/229886756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819019fbe0 con 0x7f8190102630 2024-04-25T16:37:40.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.746+0000 7f8195560700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102630 0x7f819019ad10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f819006b010 tx=0x7f818c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:40.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.746+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c014030 con 0x7f8190102630 2024-04-25T16:37:40.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.746+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818c00fb40 con 0x7f8190102630 2024-04-25T16:37:40.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.746+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c010cf0 con 0x7f8190102630 2024-04-25T16:37:40.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.746+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819019fed0 con 0x7f8190102630 2024-04-25T16:37:40.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.746+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901a77f0 con 0x7f8190102630 2024-04-25T16:37:40.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.747+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8190051190 con 0x7f8190102630 2024-04-25T16:37:40.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.749+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7f818c017030 con 0x7f8190102630 2024-04-25T16:37:40.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.750+0000 7f81867fc700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f817c0afc90 0x7f817c0b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:40.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.750+0000 7f8194d5f700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f817c0afc90 0x7f817c0b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:40.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.750+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f818c0de660 con 0x7f8190102630 2024-04-25T16:37:40.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.750+0000 7f8194d5f700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f817c0afc90 0x7f817c0b21a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8190107cd0 tx=0x7f818000b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:40.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.755+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f818c0a70c0 con 0x7f8190102630 2024-04-25T16:37:40.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.956+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f819002e510 con 0x7f8190102630 2024-04-25T16:37:40.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.956+0000 7f81867fc700 1 -- 172.21.15.112:0/229886756 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f818c0a69c0 con 0x7f8190102630 2024-04-25T16:37:40.955 INFO:teuthology.orchestra.run.smithi112.stdout:34359738453 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f817c0afc90 msgr2=0x7f817c0b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f817c0afc90 0x7f817c0b21a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8190107cd0 tx=0x7f818000b480 comp rx=0 tx=0).stop 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102630 msgr2=0x7f819019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102630 0x7f819019ad10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f819006b010 tx=0x7f818c00da00 comp rx=0 tx=0).stop 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 shutdown_connections 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f817c0afc90 0x7f817c0b21a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81901091a0 0x7f819019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8190102630 0x7f819019ad10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 --2- 172.21.15.112:0/229886756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8190102f80 0x7f819019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 >> 172.21.15.112:0/229886756 conn(0x7f81900fe3f0 msgr2=0x7f819010a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 shutdown_connections 2024-04-25T16:37:40.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:40.958+0000 7f81977c4700 1 -- 172.21.15.112:0/229886756 wait complete. 2024-04-25T16:37:41.151 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986235 got 133143986241 for osd.4 2024-04-25T16:37:41.152 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:41.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:41 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/229886756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T16:37:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:41 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/229886756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T16:37:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:41 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/229886756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T16:37:41.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738448 got 34359738453 for osd.0 2024-04-25T16:37:41.749 DEBUG:teuthology.parallel:result is None 2024-04-25T16:37:41.749 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T16:37:41.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T16:37:42.284 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:42 smithi112 ceph-mon[29122]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:37:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:42 smithi133 ceph-mon[32691]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:37:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:42 smithi116 ceph-mon[32712]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:37:42.556 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 -- 172.21.15.112:0/2058982681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 msgr2=0x7fa5941091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 --2- 172.21.15.112:0/2058982681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941091f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa5840099c0 tx=0x7fa584009cd0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 -- 172.21.15.112:0/2058982681 shutdown_connections 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 --2- 172.21.15.112:0/2058982681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 0x7fa594111090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 --2- 172.21.15.112:0/2058982681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 0x7fa5941048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 --2- 172.21.15.112:0/2058982681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941091f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 -- 172.21.15.112:0/2058982681 >> 172.21.15.112:0/2058982681 conn(0x7fa5940ffda0 msgr2=0x7fa5941021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:44.234 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:44 smithi112 ceph-mon[29122]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:37:44.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 -- 172.21.15.112:0/2058982681 shutdown_connections 2024-04-25T16:37:44.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.234+0000 7fa59be7e700 1 -- 172.21.15.112:0/2058982681 wait complete. 2024-04-25T16:37:44.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.235+0000 7fa59be7e700 1 Processor -- start 2024-04-25T16:37:44.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.235+0000 7fa59be7e700 1 -- start start 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59be7e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 0x7fa5941a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59be7e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59be7e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 0x7fa59419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59be7e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa594114360 con 0x7fa594108de0 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59be7e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941141e0 con 0x7fa5941044b0 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59be7e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa5941144e0 con 0x7fa594109730 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599c1a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 0x7fa5941a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941a0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49498/0 (socket says 172.21.15.112:49498) 2024-04-25T16:37:44.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 -- 172.21.15.112:0/1057686838 learned_addr learned my addr 172.21.15.112:0/1057686838 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59a41b700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 0x7fa59419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 -- 172.21.15.112:0/1057686838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 msgr2=0x7fa59419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 0x7fa59419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 -- 172.21.15.112:0/1057686838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 msgr2=0x7fa5941a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 0x7fa5941a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 -- 172.21.15.112:0/1057686838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59419b1e0 con 0x7fa594108de0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa59a41b700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 0x7fa59419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599c1a700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 0x7fa5941a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.236+0000 7fa599419700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941a0e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5840045e0 tx=0x7fa584004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.237+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584026070 con 0x7fa594108de0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.237+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa58400f460 con 0x7fa594108de0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.237+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59419b470 con 0x7fa594108de0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.237+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa584009e60 con 0x7fa594108de0 2024-04-25T16:37:44.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.237+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5941a7770 con 0x7fa594108de0 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.238+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa578005330 con 0x7fa594108de0 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.239+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 17) ==== 147572+0+0 (secure 0 0 0) 0x7fa584029070 con 0x7fa594108de0 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.240+0000 7fa58affd700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5800afc90 0x7fa5800b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.240+0000 7fa599c1a700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5800afc90 0x7fa5800b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.240+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa5840df730 con 0x7fa594108de0 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.240+0000 7fa599c1a700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5800afc90 0x7fa5800b21a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa59419bc40 tx=0x7fa590009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:44.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.244+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa584013070 con 0x7fa594108de0 2024-04-25T16:37:44.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:44 smithi133 ceph-mon[32691]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:37:44.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.471+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa578000bc0 con 0x7fa5800afc90 2024-04-25T16:37:44.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.478+0000 7fa58affd700 1 -- 172.21.15.112:0/1057686838 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258962 (secure 0 0 0) 0x7fa578000bc0 con 0x7fa5800afc90 2024-04-25T16:37:44.480 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:44.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.480+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5800afc90 msgr2=0x7fa5800b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.480+0000 7fa59be7e700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5800afc90 0x7fa5800b21a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa59419bc40 tx=0x7fa590009260 comp rx=0 tx=0).stop 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.480+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 msgr2=0x7fa5941a0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.480+0000 7fa59be7e700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941a0e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa5840045e0 tx=0x7fa584004610 comp rx=0 tx=0).stop 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 shutdown_connections 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594109730 0x7fa59419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa5800afc90 0x7fa5800b21a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5941044b0 0x7fa5941a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 --2- 172.21.15.112:0/1057686838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa594108de0 0x7fa5941a0e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 >> 172.21.15.112:0/1057686838 conn(0x7fa5940ffda0 msgr2=0x7fa594108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 shutdown_connections 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:44.481+0000 7fa59be7e700 1 -- 172.21.15.112:0/1057686838 wait complete. 2024-04-25T16:37:44.483 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T16:37:44.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:44 smithi116 ceph-mon[32712]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:37:45.108 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-25T16:37:43.100370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":523,"num_read_kb":354,"num_write":281,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":346968,"kb_used_data":10212,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594856,"statfs":{"total":1151940427776,"available":1151585132544,"internally_reserved":0,"allocated":10457088,"data_stored":6779499,"data_compressed":182058,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003315"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211566+0000","last_change":"2024-04-25T16:36:26.988707+0000","last_active":"2024-04-25T16:37:16.211566+0000","last_peered":"2024-04-25T16:37:16.211566+0000","last_clean":"2024-04-25T16:37:16.211566+0000","last_became_active":"2024-04-25T16:36:26.988302+0000","last_became_peered":"2024-04-25T16:36:26.988302+0000","last_unstale":"2024-04-25T16:37:16.211566+0000","last_undegraded":"2024-04-25T16:37:16.211566+0000","last_fullsized":"2024-04-25T16:37:16.211566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:44.074090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207541+0000","last_change":"2024-04-25T16:36:26.988147+0000","last_active":"2024-04-25T16:37:16.207541+0000","last_peered":"2024-04-25T16:37:16.207541+0000","last_clean":"2024-04-25T16:37:16.207541+0000","last_became_active":"2024-04-25T16:36:26.987695+0000","last_became_peered":"2024-04-25T16:36:26.987695+0000","last_unstale":"2024-04-25T16:37:16.207541+0000","last_undegraded":"2024-04-25T16:37:16.207541+0000","last_fullsized":"2024-04-25T16:37:16.207541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:09.330235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207145+0000","last_change":"2024-04-25T16:36:26.986601+0000","last_active":"2024-04-25T16:37:16.207145+0000","last_peered":"2024-04-25T16:37:16.207145+0000","last_clean":"2024-04-25T16:37:16.207145+0000","last_became_active":"2024-04-25T16:36:26.985431+0000","last_became_peered":"2024-04-25T16:36:26.985431+0000","last_unstale":"2024-04-25T16:37:16.207145+0000","last_undegraded":"2024-04-25T16:37:16.207145+0000","last_fullsized":"2024-04-25T16:37:16.207145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:09:52.756550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204375+0000","last_change":"2024-04-25T16:36:26.985007+0000","last_active":"2024-04-25T16:37:16.204375+0000","last_peered":"2024-04-25T16:37:16.204375+0000","last_clean":"2024-04-25T16:37:16.204375+0000","last_became_active":"2024-04-25T16:36:26.983000+0000","last_became_peered":"2024-04-25T16:36:26.983000+0000","last_unstale":"2024-04-25T16:37:16.204375+0000","last_undegraded":"2024-04-25T16:37:16.204375+0000","last_fullsized":"2024-04-25T16:37:16.204375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:02.324409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206930+0000","last_change":"2024-04-25T16:36:26.970714+0000","last_active":"2024-04-25T16:37:16.206930+0000","last_peered":"2024-04-25T16:37:16.206930+0000","last_clean":"2024-04-25T16:37:16.206930+0000","last_became_active":"2024-04-25T16:36:26.970323+0000","last_became_peered":"2024-04-25T16:36:26.970323+0000","last_unstale":"2024-04-25T16:37:16.206930+0000","last_undegraded":"2024-04-25T16:37:16.206930+0000","last_fullsized":"2024-04-25T16:37:16.206930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:41.113178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314698+0000","last_change":"2024-04-25T16:36:26.988388+0000","last_active":"2024-04-25T16:37:16.314698+0000","last_peered":"2024-04-25T16:37:16.314698+0000","last_clean":"2024-04-25T16:37:16.314698+0000","last_became_active":"2024-04-25T16:36:26.988090+0000","last_became_peered":"2024-04-25T16:36:26.988090+0000","last_unstale":"2024-04-25T16:37:16.314698+0000","last_undegraded":"2024-04-25T16:37:16.314698+0000","last_fullsized":"2024-04-25T16:37:16.314698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:19.666488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210381+0000","last_change":"2024-04-25T16:36:26.971244+0000","last_active":"2024-04-25T16:37:16.210381+0000","last_peered":"2024-04-25T16:37:16.210381+0000","last_clean":"2024-04-25T16:37:16.210381+0000","last_became_active":"2024-04-25T16:36:26.970900+0000","last_became_peered":"2024-04-25T16:36:26.970900+0000","last_unstale":"2024-04-25T16:37:16.210381+0000","last_undegraded":"2024-04-25T16:37:16.210381+0000","last_fullsized":"2024-04-25T16:37:16.210381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:46.434230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366317+0000","last_change":"2024-04-25T16:36:26.982614+0000","last_active":"2024-04-25T16:37:16.366317+0000","last_peered":"2024-04-25T16:37:16.366317+0000","last_clean":"2024-04-25T16:37:16.366317+0000","last_became_active":"2024-04-25T16:36:26.981911+0000","last_became_peered":"2024-04-25T16:36:26.981911+0000","last_unstale":"2024-04-25T16:37:16.366317+0000","last_undegraded":"2024-04-25T16:37:16.366317+0000","last_fullsized":"2024-04-25T16:37:16.366317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:36.186614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.212825+0000","last_change":"2024-04-25T16:37:16.212825+0000","last_active":"2024-04-25T16:37:16.212825+0000","last_peered":"2024-04-25T16:37:16.212825+0000","last_clean":"2024-04-25T16:37:16.212825+0000","last_became_active":"2024-04-25T16:37:16.212387+0000","last_became_peered":"2024-04-25T16:37:16.212387+0000","last_unstale":"2024-04-25T16:37:16.212825+0000","last_undegraded":"2024-04-25T16:37:16.212825+0000","last_fullsized":"2024-04-25T16:37:16.212825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:49.060551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.315082+0000","last_change":"2024-04-25T16:36:26.989299+0000","last_active":"2024-04-25T16:37:16.315082+0000","last_peered":"2024-04-25T16:37:16.315082+0000","last_clean":"2024-04-25T16:37:16.315082+0000","last_became_active":"2024-04-25T16:36:26.988826+0000","last_became_peered":"2024-04-25T16:36:26.988826+0000","last_unstale":"2024-04-25T16:37:16.315082+0000","last_undegraded":"2024-04-25T16:37:16.315082+0000","last_fullsized":"2024-04-25T16:37:16.315082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:40.549866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211051+0000","last_change":"2024-04-25T16:36:26.971847+0000","last_active":"2024-04-25T16:37:16.211051+0000","last_peered":"2024-04-25T16:37:16.211051+0000","last_clean":"2024-04-25T16:37:16.211051+0000","last_became_active":"2024-04-25T16:36:26.971417+0000","last_became_peered":"2024-04-25T16:36:26.971417+0000","last_unstale":"2024-04-25T16:37:16.211051+0000","last_undegraded":"2024-04-25T16:37:16.211051+0000","last_fullsized":"2024-04-25T16:37:16.211051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:55.075777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204499+0000","last_change":"2024-04-25T16:36:26.985714+0000","last_active":"2024-04-25T16:37:16.204499+0000","last_peered":"2024-04-25T16:37:16.204499+0000","last_clean":"2024-04-25T16:37:16.204499+0000","last_became_active":"2024-04-25T16:36:26.985402+0000","last_became_peered":"2024-04-25T16:36:26.985402+0000","last_unstale":"2024-04-25T16:37:16.204499+0000","last_undegraded":"2024-04-25T16:37:16.204499+0000","last_fullsized":"2024-04-25T16:37:16.204499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:32.953895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206083+0000","last_change":"2024-04-25T16:36:26.976917+0000","last_active":"2024-04-25T16:37:16.206083+0000","last_peered":"2024-04-25T16:37:16.206083+0000","last_clean":"2024-04-25T16:37:16.206083+0000","last_became_active":"2024-04-25T16:36:26.976452+0000","last_became_peered":"2024-04-25T16:36:26.976452+0000","last_unstale":"2024-04-25T16:37:16.206083+0000","last_undegraded":"2024-04-25T16:37:16.206083+0000","last_fullsized":"2024-04-25T16:37:16.206083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:45.530249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207063+0000","last_change":"2024-04-25T16:36:26.987433+0000","last_active":"2024-04-25T16:37:16.207063+0000","last_peered":"2024-04-25T16:37:16.207063+0000","last_clean":"2024-04-25T16:37:16.207063+0000","last_became_active":"2024-04-25T16:36:26.987106+0000","last_became_peered":"2024-04-25T16:36:26.987106+0000","last_unstale":"2024-04-25T16:37:16.207063+0000","last_undegraded":"2024-04-25T16:37:16.207063+0000","last_fullsized":"2024-04-25T16:37:16.207063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:33.436034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206750+0000","last_change":"2024-04-25T16:36:26.984530+0000","last_active":"2024-04-25T16:37:16.206750+0000","last_peered":"2024-04-25T16:37:16.206750+0000","last_clean":"2024-04-25T16:37:16.206750+0000","last_became_active":"2024-04-25T16:36:26.984220+0000","last_became_peered":"2024-04-25T16:36:26.984220+0000","last_unstale":"2024-04-25T16:37:16.206750+0000","last_undegraded":"2024-04-25T16:37:16.206750+0000","last_fullsized":"2024-04-25T16:37:16.206750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:06.345465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206097+0000","last_change":"2024-04-25T16:36:26.980249+0000","last_active":"2024-04-25T16:37:16.206097+0000","last_peered":"2024-04-25T16:37:16.206097+0000","last_clean":"2024-04-25T16:37:16.206097+0000","last_became_active":"2024-04-25T16:36:26.979861+0000","last_became_peered":"2024-04-25T16:36:26.979861+0000","last_unstale":"2024-04-25T16:37:16.206097+0000","last_undegraded":"2024-04-25T16:37:16.206097+0000","last_fullsized":"2024-04-25T16:37:16.206097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:26.753789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211738+0000","last_change":"2024-04-25T16:36:26.988022+0000","last_active":"2024-04-25T16:37:16.211738+0000","last_peered":"2024-04-25T16:37:16.211738+0000","last_clean":"2024-04-25T16:37:16.211738+0000","last_became_active":"2024-04-25T16:36:26.987609+0000","last_became_peered":"2024-04-25T16:36:26.987609+0000","last_unstale":"2024-04-25T16:37:16.211738+0000","last_undegraded":"2024-04-25T16:37:16.211738+0000","last_fullsized":"2024-04-25T16:37:16.211738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.819517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214455+0000","last_change":"2024-04-25T16:36:26.978511+0000","last_active":"2024-04-25T16:37:16.214455+0000","last_peered":"2024-04-25T16:37:16.214455+0000","last_clean":"2024-04-25T16:37:16.214455+0000","last_became_active":"2024-04-25T16:36:26.978180+0000","last_became_peered":"2024-04-25T16:36:26.978180+0000","last_unstale":"2024-04-25T16:37:16.214455+0000","last_undegraded":"2024-04-25T16:37:16.214455+0000","last_fullsized":"2024-04-25T16:37:16.214455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:58.931649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217509+0000","last_change":"2024-04-25T16:37:16.217509+0000","last_active":"2024-04-25T16:37:16.217509+0000","last_peered":"2024-04-25T16:37:16.217509+0000","last_clean":"2024-04-25T16:37:16.217509+0000","last_became_active":"2024-04-25T16:37:16.217018+0000","last_became_peered":"2024-04-25T16:37:16.217018+0000","last_unstale":"2024-04-25T16:37:16.217509+0000","last_undegraded":"2024-04-25T16:37:16.217509+0000","last_fullsized":"2024-04-25T16:37:16.217509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:43.703130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214254+0000","last_change":"2024-04-25T16:36:26.216356+0000","last_active":"2024-04-25T16:37:16.214254+0000","last_peered":"2024-04-25T16:37:16.214254+0000","last_clean":"2024-04-25T16:37:16.214254+0000","last_became_active":"2024-04-25T16:36:26.215935+0000","last_became_peered":"2024-04-25T16:36:26.215935+0000","last_unstale":"2024-04-25T16:37:16.214254+0000","last_undegraded":"2024-04-25T16:37:16.214254+0000","last_fullsized":"2024-04-25T16:37:16.214254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:58.186003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366688+0000","last_change":"2024-04-25T16:36:26.981307+0000","last_active":"2024-04-25T16:37:16.366688+0000","last_peered":"2024-04-25T16:37:16.366688+0000","last_clean":"2024-04-25T16:37:16.366688+0000","last_became_active":"2024-04-25T16:36:26.980973+0000","last_became_peered":"2024-04-25T16:36:26.980973+0000","last_unstale":"2024-04-25T16:37:16.366688+0000","last_undegraded":"2024-04-25T16:37:16.366688+0000","last_fullsized":"2024-04-25T16:37:16.366688+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:45.469266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366820+0000","last_change":"2024-04-25T16:36:26.218839+0000","last_active":"2024-04-25T16:37:16.366820+0000","last_peered":"2024-04-25T16:37:16.366820+0000","last_clean":"2024-04-25T16:37:16.366820+0000","last_became_active":"2024-04-25T16:36:26.218487+0000","last_became_peered":"2024-04-25T16:36:26.218487+0000","last_unstale":"2024-04-25T16:37:16.366820+0000","last_undegraded":"2024-04-25T16:37:16.366820+0000","last_fullsized":"2024-04-25T16:37:16.366820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:05.164084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207745+0000","last_change":"2024-04-25T16:36:26.977277+0000","last_active":"2024-04-25T16:37:16.207745+0000","last_peered":"2024-04-25T16:37:16.207745+0000","last_clean":"2024-04-25T16:37:16.207745+0000","last_became_active":"2024-04-25T16:36:26.974805+0000","last_became_peered":"2024-04-25T16:36:26.974805+0000","last_unstale":"2024-04-25T16:37:16.207745+0000","last_undegraded":"2024-04-25T16:37:16.207745+0000","last_fullsized":"2024-04-25T16:37:16.207745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:45.389875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.208108+0000","last_change":"2024-04-25T16:36:26.215757+0000","last_active":"2024-04-25T16:37:16.208108+0000","last_peered":"2024-04-25T16:37:16.208108+0000","last_clean":"2024-04-25T16:37:16.208108+0000","last_became_active":"2024-04-25T16:36:26.215424+0000","last_became_peered":"2024-04-25T16:36:26.215424+0000","last_unstale":"2024-04-25T16:37:16.208108+0000","last_undegraded":"2024-04-25T16:37:16.208108+0000","last_fullsized":"2024-04-25T16:37:16.208108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:23.094599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206512+0000","last_change":"2024-04-25T16:36:26.978552+0000","last_active":"2024-04-25T16:37:16.206512+0000","last_peered":"2024-04-25T16:37:16.206512+0000","last_clean":"2024-04-25T16:37:16.206512+0000","last_became_active":"2024-04-25T16:36:26.978185+0000","last_became_peered":"2024-04-25T16:36:26.978185+0000","last_unstale":"2024-04-25T16:37:16.206512+0000","last_undegraded":"2024-04-25T16:37:16.206512+0000","last_fullsized":"2024-04-25T16:37:16.206512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:19.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207606+0000","last_change":"2024-04-25T16:36:26.207361+0000","last_active":"2024-04-25T16:37:16.207606+0000","last_peered":"2024-04-25T16:37:16.207606+0000","last_clean":"2024-04-25T16:37:16.207606+0000","last_became_active":"2024-04-25T16:36:26.206968+0000","last_became_peered":"2024-04-25T16:36:26.206968+0000","last_unstale":"2024-04-25T16:37:16.207606+0000","last_undegraded":"2024-04-25T16:37:16.207606+0000","last_fullsized":"2024-04-25T16:37:16.207606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:54.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204182+0000","last_change":"2024-04-25T16:36:26.979723+0000","last_active":"2024-04-25T16:37:16.204182+0000","last_peered":"2024-04-25T16:37:16.204182+0000","last_clean":"2024-04-25T16:37:16.204182+0000","last_became_active":"2024-04-25T16:36:26.979377+0000","last_became_peered":"2024-04-25T16:36:26.979377+0000","last_unstale":"2024-04-25T16:37:16.204182+0000","last_undegraded":"2024-04-25T16:37:16.204182+0000","last_fullsized":"2024-04-25T16:37:16.204182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:50.312759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":100,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:41.351181+0000","last_change":"2024-04-25T16:36:26.217074+0000","last_active":"2024-04-25T16:37:41.351181+0000","last_peered":"2024-04-25T16:37:41.351181+0000","last_clean":"2024-04-25T16:37:41.351181+0000","last_became_active":"2024-04-25T16:36:26.216686+0000","last_became_peered":"2024-04-25T16:36:26.216686+0000","last_unstale":"2024-04-25T16:37:41.351181+0000","last_undegraded":"2024-04-25T16:37:41.351181+0000","last_fullsized":"2024-04-25T16:37:41.351181+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:35.780179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.216828+0000","last_change":"2024-04-25T16:37:16.216828+0000","last_active":"2024-04-25T16:37:16.216828+0000","last_peered":"2024-04-25T16:37:16.216828+0000","last_clean":"2024-04-25T16:37:16.216828+0000","last_became_active":"2024-04-25T16:37:16.216350+0000","last_became_peered":"2024-04-25T16:37:16.216350+0000","last_unstale":"2024-04-25T16:37:16.216828+0000","last_undegraded":"2024-04-25T16:37:16.216828+0000","last_fullsized":"2024-04-25T16:37:16.216828+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:32.705434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314600+0000","last_change":"2024-04-25T16:36:26.211703+0000","last_active":"2024-04-25T16:37:16.314600+0000","last_peered":"2024-04-25T16:37:16.314600+0000","last_clean":"2024-04-25T16:37:16.314600+0000","last_became_active":"2024-04-25T16:36:26.211322+0000","last_became_peered":"2024-04-25T16:36:26.211322+0000","last_unstale":"2024-04-25T16:37:16.314600+0000","last_undegraded":"2024-04-25T16:37:16.314600+0000","last_fullsized":"2024-04-25T16:37:16.314600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:15.353208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210437+0000","last_change":"2024-04-25T16:37:16.210437+0000","last_active":"2024-04-25T16:37:16.210437+0000","last_peered":"2024-04-25T16:37:16.210437+0000","last_clean":"2024-04-25T16:37:16.210437+0000","last_became_active":"2024-04-25T16:37:16.210030+0000","last_became_peered":"2024-04-25T16:37:16.210030+0000","last_unstale":"2024-04-25T16:37:16.210437+0000","last_undegraded":"2024-04-25T16:37:16.210437+0000","last_fullsized":"2024-04-25T16:37:16.210437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.511125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206083+0000","last_change":"2024-04-25T16:36:26.217225+0000","last_active":"2024-04-25T16:37:16.206083+0000","last_peered":"2024-04-25T16:37:16.206083+0000","last_clean":"2024-04-25T16:37:16.206083+0000","last_became_active":"2024-04-25T16:36:26.216777+0000","last_became_peered":"2024-04-25T16:36:26.216777+0000","last_unstale":"2024-04-25T16:37:16.206083+0000","last_undegraded":"2024-04-25T16:37:16.206083+0000","last_fullsized":"2024-04-25T16:37:16.206083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:21.433977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366441+0000","last_change":"2024-04-25T16:36:26.970197+0000","last_active":"2024-04-25T16:37:16.366441+0000","last_peered":"2024-04-25T16:37:16.366441+0000","last_clean":"2024-04-25T16:37:16.366441+0000","last_became_active":"2024-04-25T16:36:26.969844+0000","last_became_peered":"2024-04-25T16:36:26.969844+0000","last_unstale":"2024-04-25T16:37:16.366441+0000","last_undegraded":"2024-04-25T16:37:16.366441+0000","last_fullsized":"2024-04-25T16:37:16.366441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:35.397143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.203900+0000","last_change":"2024-04-25T16:36:26.214731+0000","last_active":"2024-04-25T16:37:16.203900+0000","last_peered":"2024-04-25T16:37:16.203900+0000","last_clean":"2024-04-25T16:37:16.203900+0000","last_became_active":"2024-04-25T16:36:26.214343+0000","last_became_peered":"2024-04-25T16:36:26.214343+0000","last_unstale":"2024-04-25T16:37:16.203900+0000","last_undegraded":"2024-04-25T16:37:16.203900+0000","last_fullsized":"2024-04-25T16:37:16.203900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:08:10.156665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214350+0000","last_change":"2024-04-25T16:36:26.981458+0000","last_active":"2024-04-25T16:37:16.214350+0000","last_peered":"2024-04-25T16:37:16.214350+0000","last_clean":"2024-04-25T16:37:16.214350+0000","last_became_active":"2024-04-25T16:36:26.981116+0000","last_became_peered":"2024-04-25T16:36:26.981116+0000","last_unstale":"2024-04-25T16:37:16.214350+0000","last_undegraded":"2024-04-25T16:37:16.214350+0000","last_fullsized":"2024-04-25T16:37:16.214350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:20.549670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'3","reported_seq":90,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:37.345010+0000","last_change":"2024-04-25T16:36:26.218690+0000","last_active":"2024-04-25T16:37:37.345010+0000","last_peered":"2024-04-25T16:37:37.345010+0000","last_clean":"2024-04-25T16:37:37.345010+0000","last_became_active":"2024-04-25T16:36:26.218287+0000","last_became_peered":"2024-04-25T16:36:26.218287+0000","last_unstale":"2024-04-25T16:37:37.345010+0000","last_undegraded":"2024-04-25T16:37:37.345010+0000","last_fullsized":"2024-04-25T16:37:37.345010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:42.394413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206905+0000","last_change":"2024-04-25T16:36:26.980356+0000","last_active":"2024-04-25T16:37:16.206905+0000","last_peered":"2024-04-25T16:37:16.206905+0000","last_clean":"2024-04-25T16:37:16.206905+0000","last_became_active":"2024-04-25T16:36:26.980005+0000","last_became_peered":"2024-04-25T16:36:26.980005+0000","last_unstale":"2024-04-25T16:37:16.206905+0000","last_undegraded":"2024-04-25T16:37:16.206905+0000","last_fullsized":"2024-04-25T16:37:16.206905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:37.263655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210781+0000","last_change":"2024-04-25T16:36:26.217818+0000","last_active":"2024-04-25T16:37:16.210781+0000","last_peered":"2024-04-25T16:37:16.210781+0000","last_clean":"2024-04-25T16:37:16.210781+0000","last_became_active":"2024-04-25T16:36:26.217317+0000","last_became_peered":"2024-04-25T16:36:26.217317+0000","last_unstale":"2024-04-25T16:37:16.210781+0000","last_undegraded":"2024-04-25T16:37:16.210781+0000","last_fullsized":"2024-04-25T16:37:16.210781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:29.985735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210320+0000","last_change":"2024-04-25T16:37:16.210320+0000","last_active":"2024-04-25T16:37:16.210320+0000","last_peered":"2024-04-25T16:37:16.210320+0000","last_clean":"2024-04-25T16:37:16.210320+0000","last_became_active":"2024-04-25T16:37:16.209914+0000","last_became_peered":"2024-04-25T16:37:16.209914+0000","last_unstale":"2024-04-25T16:37:16.210320+0000","last_undegraded":"2024-04-25T16:37:16.210320+0000","last_fullsized":"2024-04-25T16:37:16.210320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:33.673346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204381+0000","last_change":"2024-04-25T16:36:26.208756+0000","last_active":"2024-04-25T16:37:16.204381+0000","last_peered":"2024-04-25T16:37:16.204381+0000","last_clean":"2024-04-25T16:37:16.204381+0000","last_became_active":"2024-04-25T16:36:26.208384+0000","last_became_peered":"2024-04-25T16:36:26.208384+0000","last_unstale":"2024-04-25T16:37:16.204381+0000","last_undegraded":"2024-04-25T16:37:16.204381+0000","last_fullsized":"2024-04-25T16:37:16.204381+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.849402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206484+0000","last_change":"2024-04-25T16:36:26.981611+0000","last_active":"2024-04-25T16:37:16.206484+0000","last_peered":"2024-04-25T16:37:16.206484+0000","last_clean":"2024-04-25T16:37:16.206484+0000","last_became_active":"2024-04-25T16:36:26.981311+0000","last_became_peered":"2024-04-25T16:36:26.981311+0000","last_unstale":"2024-04-25T16:37:16.206484+0000","last_undegraded":"2024-04-25T16:37:16.206484+0000","last_fullsized":"2024-04-25T16:37:16.206484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:05.360035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'28","reported_seq":191,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:38.346177+0000","last_change":"2024-04-25T16:36:26.214215+0000","last_active":"2024-04-25T16:37:38.346177+0000","last_peered":"2024-04-25T16:37:38.346177+0000","last_clean":"2024-04-25T16:37:38.346177+0000","last_became_active":"2024-04-25T16:36:26.213848+0000","last_became_peered":"2024-04-25T16:36:26.213848+0000","last_unstale":"2024-04-25T16:37:38.346177+0000","last_undegraded":"2024-04-25T16:37:38.346177+0000","last_fullsized":"2024-04-25T16:37:38.346177+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:32.589646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":144,"num_read_kb":131,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207704+0000","last_change":"2024-04-25T16:36:26.976087+0000","last_active":"2024-04-25T16:37:16.207704+0000","last_peered":"2024-04-25T16:37:16.207704+0000","last_clean":"2024-04-25T16:37:16.207704+0000","last_became_active":"2024-04-25T16:36:26.975726+0000","last_became_peered":"2024-04-25T16:36:26.975726+0000","last_unstale":"2024-04-25T16:37:16.207704+0000","last_undegraded":"2024-04-25T16:37:16.207704+0000","last_fullsized":"2024-04-25T16:37:16.207704+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:02.668930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.208985+0000","last_change":"2024-04-25T16:36:26.208815+0000","last_active":"2024-04-25T16:37:16.208985+0000","last_peered":"2024-04-25T16:37:16.208985+0000","last_clean":"2024-04-25T16:37:16.208985+0000","last_became_active":"2024-04-25T16:36:26.208485+0000","last_became_peered":"2024-04-25T16:36:26.208485+0000","last_unstale":"2024-04-25T16:37:16.208985+0000","last_undegraded":"2024-04-25T16:37:16.208985+0000","last_fullsized":"2024-04-25T16:37:16.208985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:11.984900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.205746+0000","last_change":"2024-04-25T16:36:26.975567+0000","last_active":"2024-04-25T16:37:16.205746+0000","last_peered":"2024-04-25T16:37:16.205746+0000","last_clean":"2024-04-25T16:37:16.205746+0000","last_became_active":"2024-04-25T16:36:26.975212+0000","last_became_peered":"2024-04-25T16:36:26.975212+0000","last_unstale":"2024-04-25T16:37:16.205746+0000","last_undegraded":"2024-04-25T16:37:16.205746+0000","last_fullsized":"2024-04-25T16:37:16.205746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:34.778011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204089+0000","last_change":"2024-04-25T16:36:26.214765+0000","last_active":"2024-04-25T16:37:16.204089+0000","last_peered":"2024-04-25T16:37:16.204089+0000","last_clean":"2024-04-25T16:37:16.204089+0000","last_became_active":"2024-04-25T16:36:26.214378+0000","last_became_peered":"2024-04-25T16:36:26.214378+0000","last_unstale":"2024-04-25T16:37:16.204089+0000","last_undegraded":"2024-04-25T16:37:16.204089+0000","last_fullsized":"2024-04-25T16:37:16.204089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:00.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210983+0000","last_change":"2024-04-25T16:36:26.978074+0000","last_active":"2024-04-25T16:37:16.210983+0000","last_peered":"2024-04-25T16:37:16.210983+0000","last_clean":"2024-04-25T16:37:16.210983+0000","last_became_active":"2024-04-25T16:36:26.977706+0000","last_became_peered":"2024-04-25T16:36:26.977706+0000","last_unstale":"2024-04-25T16:37:16.210983+0000","last_undegraded":"2024-04-25T16:37:16.210983+0000","last_fullsized":"2024-04-25T16:37:16.210983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:05.718026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'3","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:41.351062+0000","last_change":"2024-04-25T16:36:26.211526+0000","last_active":"2024-04-25T16:37:41.351062+0000","last_peered":"2024-04-25T16:37:41.351062+0000","last_clean":"2024-04-25T16:37:41.351062+0000","last_became_active":"2024-04-25T16:36:26.211134+0000","last_became_peered":"2024-04-25T16:36:26.211134+0000","last_unstale":"2024-04-25T16:37:41.351062+0000","last_undegraded":"2024-04-25T16:37:41.351062+0000","last_fullsized":"2024-04-25T16:37:41.351062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:25:39.743843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210536+0000","last_change":"2024-04-25T16:36:26.979766+0000","last_active":"2024-04-25T16:37:16.210536+0000","last_peered":"2024-04-25T16:37:16.210536+0000","last_clean":"2024-04-25T16:37:16.210536+0000","last_became_active":"2024-04-25T16:36:26.979460+0000","last_became_peered":"2024-04-25T16:36:26.979460+0000","last_unstale":"2024-04-25T16:37:16.210536+0000","last_undegraded":"2024-04-25T16:37:16.210536+0000","last_fullsized":"2024-04-25T16:37:16.210536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.957289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206765+0000","last_change":"2024-04-25T16:36:26.212112+0000","last_active":"2024-04-25T16:37:16.206765+0000","last_peered":"2024-04-25T16:37:16.206765+0000","last_clean":"2024-04-25T16:37:16.206765+0000","last_became_active":"2024-04-25T16:36:26.211746+0000","last_became_peered":"2024-04-25T16:36:26.211746+0000","last_unstale":"2024-04-25T16:37:16.206765+0000","last_undegraded":"2024-04-25T16:37:16.206765+0000","last_fullsized":"2024-04-25T16:37:16.206765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:24.682525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.203953+0000","last_change":"2024-04-25T16:36:26.980275+0000","last_active":"2024-04-25T16:37:16.203953+0000","last_peered":"2024-04-25T16:37:16.203953+0000","last_clean":"2024-04-25T16:37:16.203953+0000","last_became_active":"2024-04-25T16:36:26.979912+0000","last_became_peered":"2024-04-25T16:36:26.979912+0000","last_unstale":"2024-04-25T16:37:16.203953+0000","last_undegraded":"2024-04-25T16:37:16.203953+0000","last_fullsized":"2024-04-25T16:37:16.203953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:28.802041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207890+0000","last_change":"2024-04-25T16:36:26.205090+0000","last_active":"2024-04-25T16:37:16.207890+0000","last_peered":"2024-04-25T16:37:16.207890+0000","last_clean":"2024-04-25T16:37:16.207890+0000","last_became_active":"2024-04-25T16:36:26.204707+0000","last_became_peered":"2024-04-25T16:36:26.204707+0000","last_unstale":"2024-04-25T16:37:16.207890+0000","last_undegraded":"2024-04-25T16:37:16.207890+0000","last_fullsized":"2024-04-25T16:37:16.207890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:09:13.479241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211479+0000","last_change":"2024-04-25T16:36:26.985815+0000","last_active":"2024-04-25T16:37:16.211479+0000","last_peered":"2024-04-25T16:37:16.211479+0000","last_clean":"2024-04-25T16:37:16.211479+0000","last_became_active":"2024-04-25T16:36:26.985472+0000","last_became_peered":"2024-04-25T16:36:26.985472+0000","last_unstale":"2024-04-25T16:37:16.211479+0000","last_undegraded":"2024-04-25T16:37:16.211479+0000","last_fullsized":"2024-04-25T16:37:16.211479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:33.883655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214288+0000","last_change":"2024-04-25T16:36:26.207352+0000","last_active":"2024-04-25T16:37:16.214288+0000","last_peered":"2024-04-25T16:37:16.214288+0000","last_clean":"2024-04-25T16:37:16.214288+0000","last_became_active":"2024-04-25T16:36:26.206975+0000","last_became_peered":"2024-04-25T16:36:26.206975+0000","last_unstale":"2024-04-25T16:37:16.214288+0000","last_undegraded":"2024-04-25T16:37:16.214288+0000","last_fullsized":"2024-04-25T16:37:16.214288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:03.085034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314805+0000","last_change":"2024-04-25T16:36:26.982401+0000","last_active":"2024-04-25T16:37:16.314805+0000","last_peered":"2024-04-25T16:37:16.314805+0000","last_clean":"2024-04-25T16:37:16.314805+0000","last_became_active":"2024-04-25T16:36:26.982070+0000","last_became_peered":"2024-04-25T16:36:26.982070+0000","last_unstale":"2024-04-25T16:37:16.314805+0000","last_undegraded":"2024-04-25T16:37:16.314805+0000","last_fullsized":"2024-04-25T16:37:16.314805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.824276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210632+0000","last_change":"2024-04-25T16:36:26.214759+0000","last_active":"2024-04-25T16:37:16.210632+0000","last_peered":"2024-04-25T16:37:16.210632+0000","last_clean":"2024-04-25T16:37:16.210632+0000","last_became_active":"2024-04-25T16:36:26.214334+0000","last_became_peered":"2024-04-25T16:36:26.214334+0000","last_unstale":"2024-04-25T16:37:16.210632+0000","last_undegraded":"2024-04-25T16:37:16.210632+0000","last_fullsized":"2024-04-25T16:37:16.210632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:00.537377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314680+0000","last_change":"2024-04-25T16:36:26.988313+0000","last_active":"2024-04-25T16:37:16.314680+0000","last_peered":"2024-04-25T16:37:16.314680+0000","last_clean":"2024-04-25T16:37:16.314680+0000","last_became_active":"2024-04-25T16:36:26.987979+0000","last_became_peered":"2024-04-25T16:36:26.987979+0000","last_unstale":"2024-04-25T16:37:16.314680+0000","last_undegraded":"2024-04-25T16:37:16.314680+0000","last_fullsized":"2024-04-25T16:37:16.314680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:40.737779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204052+0000","last_change":"2024-04-25T16:36:26.982005+0000","last_active":"2024-04-25T16:37:16.204052+0000","last_peered":"2024-04-25T16:37:16.204052+0000","last_clean":"2024-04-25T16:37:16.204052+0000","last_became_active":"2024-04-25T16:36:26.981619+0000","last_became_peered":"2024-04-25T16:36:26.981619+0000","last_unstale":"2024-04-25T16:37:16.204052+0000","last_undegraded":"2024-04-25T16:37:16.204052+0000","last_fullsized":"2024-04-25T16:37:16.204052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:49.555114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'4","reported_seq":91,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:37.345166+0000","last_change":"2024-04-25T16:36:26.207188+0000","last_active":"2024-04-25T16:37:37.345166+0000","last_peered":"2024-04-25T16:37:37.345166+0000","last_clean":"2024-04-25T16:37:37.345166+0000","last_became_active":"2024-04-25T16:36:26.206497+0000","last_became_peered":"2024-04-25T16:36:26.206497+0000","last_unstale":"2024-04-25T16:37:37.345166+0000","last_undegraded":"2024-04-25T16:37:37.345166+0000","last_fullsized":"2024-04-25T16:37:37.345166+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:45.213263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366880+0000","last_change":"2024-04-25T16:36:26.980524+0000","last_active":"2024-04-25T16:37:16.366880+0000","last_peered":"2024-04-25T16:37:16.366880+0000","last_clean":"2024-04-25T16:37:16.366880+0000","last_became_active":"2024-04-25T16:36:26.980195+0000","last_became_peered":"2024-04-25T16:36:26.980195+0000","last_unstale":"2024-04-25T16:37:16.366880+0000","last_undegraded":"2024-04-25T16:37:16.366880+0000","last_fullsized":"2024-04-25T16:37:16.366880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:25.623403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204773+0000","last_change":"2024-04-25T16:36:26.208423+0000","last_active":"2024-04-25T16:37:16.204773+0000","last_peered":"2024-04-25T16:37:16.204773+0000","last_clean":"2024-04-25T16:37:16.204773+0000","last_became_active":"2024-04-25T16:36:26.208045+0000","last_became_peered":"2024-04-25T16:36:26.208045+0000","last_unstale":"2024-04-25T16:37:16.204773+0000","last_undegraded":"2024-04-25T16:37:16.204773+0000","last_fullsized":"2024-04-25T16:37:16.204773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:47.727762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210324+0000","last_change":"2024-04-25T16:35:10.871536+0000","last_active":"2024-04-25T16:37:16.210324+0000","last_peered":"2024-04-25T16:37:16.210324+0000","last_clean":"2024-04-25T16:37:16.210324+0000","last_became_active":"2024-04-25T16:35:10.866041+0000","last_became_peered":"2024-04-25T16:35:10.866041+0000","last_unstale":"2024-04-25T16:37:16.210324+0000","last_undegraded":"2024-04-25T16:37:16.210324+0000","last_fullsized":"2024-04-25T16:37:16.210324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_clean_scrub_stamp":"2024-04-25T16:31:41.248932+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:45.476632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217426+0000","last_change":"2024-04-25T16:37:16.217426+0000","last_active":"2024-04-25T16:37:16.217426+0000","last_peered":"2024-04-25T16:37:16.217426+0000","last_clean":"2024-04-25T16:37:16.217426+0000","last_became_active":"2024-04-25T16:37:16.216994+0000","last_became_peered":"2024-04-25T16:37:16.216994+0000","last_unstale":"2024-04-25T16:37:16.217426+0000","last_undegraded":"2024-04-25T16:37:16.217426+0000","last_fullsized":"2024-04-25T16:37:16.217426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:36.822278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210299+0000","last_change":"2024-04-25T16:36:26.219000+0000","last_active":"2024-04-25T16:37:16.210299+0000","last_peered":"2024-04-25T16:37:16.210299+0000","last_clean":"2024-04-25T16:37:16.210299+0000","last_became_active":"2024-04-25T16:36:26.218599+0000","last_became_peered":"2024-04-25T16:36:26.218599+0000","last_unstale":"2024-04-25T16:37:16.210299+0000","last_undegraded":"2024-04-25T16:37:16.210299+0000","last_fullsized":"2024-04-25T16:37:16.210299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:51.186499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.220811+0000","last_change":"2024-04-25T16:37:16.220811+0000","last_active":"2024-04-25T16:37:16.220811+0000","last_peered":"2024-04-25T16:37:16.220811+0000","last_clean":"2024-04-25T16:37:16.220811+0000","last_became_active":"2024-04-25T16:37:16.217208+0000","last_became_peered":"2024-04-25T16:37:16.217208+0000","last_unstale":"2024-04-25T16:37:16.220811+0000","last_undegraded":"2024-04-25T16:37:16.220811+0000","last_fullsized":"2024-04-25T16:37:16.220811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:56.658270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206074+0000","last_change":"2024-04-25T16:36:26.981916+0000","last_active":"2024-04-25T16:37:16.206074+0000","last_peered":"2024-04-25T16:37:16.206074+0000","last_clean":"2024-04-25T16:37:16.206074+0000","last_became_active":"2024-04-25T16:36:26.978975+0000","last_became_peered":"2024-04-25T16:36:26.978975+0000","last_unstale":"2024-04-25T16:37:16.206074+0000","last_undegraded":"2024-04-25T16:37:16.206074+0000","last_fullsized":"2024-04-25T16:37:16.206074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:43.748231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214513+0000","last_change":"2024-04-25T16:36:26.221399+0000","last_active":"2024-04-25T16:37:16.214513+0000","last_peered":"2024-04-25T16:37:16.214513+0000","last_clean":"2024-04-25T16:37:16.214513+0000","last_became_active":"2024-04-25T16:36:26.220986+0000","last_became_peered":"2024-04-25T16:36:26.220986+0000","last_unstale":"2024-04-25T16:37:16.214513+0000","last_undegraded":"2024-04-25T16:37:16.214513+0000","last_fullsized":"2024-04-25T16:37:16.214513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:59.793148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366673+0000","last_change":"2024-04-25T16:36:26.974784+0000","last_active":"2024-04-25T16:37:16.366673+0000","last_peered":"2024-04-25T16:37:16.366673+0000","last_clean":"2024-04-25T16:37:16.366673+0000","last_became_active":"2024-04-25T16:36:26.974439+0000","last_became_peered":"2024-04-25T16:36:26.974439+0000","last_unstale":"2024-04-25T16:37:16.366673+0000","last_undegraded":"2024-04-25T16:37:16.366673+0000","last_fullsized":"2024-04-25T16:37:16.366673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:30.552997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204519+0000","last_change":"2024-04-25T16:36:26.212892+0000","last_active":"2024-04-25T16:37:16.204519+0000","last_peered":"2024-04-25T16:37:16.204519+0000","last_clean":"2024-04-25T16:37:16.204519+0000","last_became_active":"2024-04-25T16:36:26.212443+0000","last_became_peered":"2024-04-25T16:36:26.212443+0000","last_unstale":"2024-04-25T16:37:16.204519+0000","last_undegraded":"2024-04-25T16:37:16.204519+0000","last_fullsized":"2024-04-25T16:37:16.204519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:25.168042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.367884+0000","last_change":"2024-04-25T16:37:16.367884+0000","last_active":"2024-04-25T16:37:16.367884+0000","last_peered":"2024-04-25T16:37:16.367884+0000","last_clean":"2024-04-25T16:37:16.367884+0000","last_became_active":"2024-04-25T16:37:16.367470+0000","last_became_peered":"2024-04-25T16:37:16.367470+0000","last_unstale":"2024-04-25T16:37:16.367884+0000","last_undegraded":"2024-04-25T16:37:16.367884+0000","last_fullsized":"2024-04-25T16:37:16.367884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:46.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206232+0000","last_change":"2024-04-25T16:36:26.212141+0000","last_active":"2024-04-25T16:37:16.206232+0000","last_peered":"2024-04-25T16:37:16.206232+0000","last_clean":"2024-04-25T16:37:16.206232+0000","last_became_active":"2024-04-25T16:36:26.211554+0000","last_became_peered":"2024-04-25T16:36:26.211554+0000","last_unstale":"2024-04-25T16:37:16.206232+0000","last_undegraded":"2024-04-25T16:37:16.206232+0000","last_fullsized":"2024-04-25T16:37:16.206232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:43.970994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206741+0000","last_change":"2024-04-25T16:36:26.978201+0000","last_active":"2024-04-25T16:37:16.206741+0000","last_peered":"2024-04-25T16:37:16.206741+0000","last_clean":"2024-04-25T16:37:16.206741+0000","last_became_active":"2024-04-25T16:36:26.977805+0000","last_became_peered":"2024-04-25T16:36:26.977805+0000","last_unstale":"2024-04-25T16:37:16.206741+0000","last_undegraded":"2024-04-25T16:37:16.206741+0000","last_fullsized":"2024-04-25T16:37:16.206741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:23.852029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366460+0000","last_change":"2024-04-25T16:36:26.211558+0000","last_active":"2024-04-25T16:37:16.366460+0000","last_peered":"2024-04-25T16:37:16.366460+0000","last_clean":"2024-04-25T16:37:16.366460+0000","last_became_active":"2024-04-25T16:36:26.211191+0000","last_became_peered":"2024-04-25T16:36:26.211191+0000","last_unstale":"2024-04-25T16:37:16.366460+0000","last_undegraded":"2024-04-25T16:37:16.366460+0000","last_fullsized":"2024-04-25T16:37:16.366460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:54.031297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211617+0000","last_change":"2024-04-25T16:36:26.987439+0000","last_active":"2024-04-25T16:37:16.211617+0000","last_peered":"2024-04-25T16:37:16.211617+0000","last_clean":"2024-04-25T16:37:16.211617+0000","last_became_active":"2024-04-25T16:36:26.986998+0000","last_became_peered":"2024-04-25T16:36:26.986998+0000","last_unstale":"2024-04-25T16:37:16.211617+0000","last_undegraded":"2024-04-25T16:37:16.211617+0000","last_fullsized":"2024-04-25T16:37:16.211617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.057303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.209127+0000","last_change":"2024-04-25T16:36:26.206662+0000","last_active":"2024-04-25T16:37:16.209127+0000","last_peered":"2024-04-25T16:37:16.209127+0000","last_clean":"2024-04-25T16:37:16.209127+0000","last_became_active":"2024-04-25T16:36:26.206237+0000","last_became_peered":"2024-04-25T16:36:26.206237+0000","last_unstale":"2024-04-25T16:37:16.209127+0000","last_undegraded":"2024-04-25T16:37:16.209127+0000","last_fullsized":"2024-04-25T16:37:16.209127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:53.602637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366678+0000","last_change":"2024-04-25T16:36:26.983837+0000","last_active":"2024-04-25T16:37:16.366678+0000","last_peered":"2024-04-25T16:37:16.366678+0000","last_clean":"2024-04-25T16:37:16.366678+0000","last_became_active":"2024-04-25T16:36:26.983513+0000","last_became_peered":"2024-04-25T16:36:26.983513+0000","last_unstale":"2024-04-25T16:37:16.366678+0000","last_undegraded":"2024-04-25T16:37:16.366678+0000","last_fullsized":"2024-04-25T16:37:16.366678+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.070866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217010+0000","last_change":"2024-04-25T16:37:16.217010+0000","last_active":"2024-04-25T16:37:16.217010+0000","last_peered":"2024-04-25T16:37:16.217010+0000","last_clean":"2024-04-25T16:37:16.217010+0000","last_became_active":"2024-04-25T16:37:16.212152+0000","last_became_peered":"2024-04-25T16:37:16.212152+0000","last_unstale":"2024-04-25T16:37:16.217010+0000","last_undegraded":"2024-04-25T16:37:16.217010+0000","last_fullsized":"2024-04-25T16:37:16.217010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:07.095559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210549+0000","last_change":"2024-04-25T16:36:26.970649+0000","last_active":"2024-04-25T16:37:16.210549+0000","last_peered":"2024-04-25T16:37:16.210549+0000","last_clean":"2024-04-25T16:37:16.210549+0000","last_became_active":"2024-04-25T16:36:26.970266+0000","last_became_peered":"2024-04-25T16:36:26.970266+0000","last_unstale":"2024-04-25T16:37:16.210549+0000","last_undegraded":"2024-04-25T16:37:16.210549+0000","last_fullsized":"2024-04-25T16:37:16.210549+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:44.055877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.209452+0000","last_change":"2024-04-25T16:36:26.212419+0000","last_active":"2024-04-25T16:37:16.209452+0000","last_peered":"2024-04-25T16:37:16.209452+0000","last_clean":"2024-04-25T16:37:16.209452+0000","last_became_active":"2024-04-25T16:36:26.211807+0000","last_became_peered":"2024-04-25T16:36:26.211807+0000","last_unstale":"2024-04-25T16:37:16.209452+0000","last_undegraded":"2024-04-25T16:37:16.209452+0000","last_fullsized":"2024-04-25T16:37:16.209452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:02.651503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211233+0000","last_change":"2024-04-25T16:36:26.970533+0000","last_active":"2024-04-25T16:37:16.211233+0000","last_peered":"2024-04-25T16:37:16.211233+0000","last_clean":"2024-04-25T16:37:16.211233+0000","last_became_active":"2024-04-25T16:36:26.970194+0000","last_became_peered":"2024-04-25T16:36:26.970194+0000","last_unstale":"2024-04-25T16:37:16.211233+0000","last_undegraded":"2024-04-25T16:37:16.211233+0000","last_fullsized":"2024-04-25T16:37:16.211233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:41.798930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204345+0000","last_change":"2024-04-25T16:36:26.210227+0000","last_active":"2024-04-25T16:37:16.204345+0000","last_peered":"2024-04-25T16:37:16.204345+0000","last_clean":"2024-04-25T16:37:16.204345+0000","last_became_active":"2024-04-25T16:36:26.209854+0000","last_became_peered":"2024-04-25T16:36:26.209854+0000","last_unstale":"2024-04-25T16:37:16.204345+0000","last_undegraded":"2024-04-25T16:37:16.204345+0000","last_fullsized":"2024-04-25T16:37:16.204345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:48.542031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210738+0000","last_change":"2024-04-25T16:36:26.971278+0000","last_active":"2024-04-25T16:37:16.210738+0000","last_peered":"2024-04-25T16:37:16.210738+0000","last_clean":"2024-04-25T16:37:16.210738+0000","last_became_active":"2024-04-25T16:36:26.970959+0000","last_became_peered":"2024-04-25T16:36:26.970959+0000","last_unstale":"2024-04-25T16:37:16.210738+0000","last_undegraded":"2024-04-25T16:37:16.210738+0000","last_fullsized":"2024-04-25T16:37:16.210738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:24.886523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206320+0000","last_change":"2024-04-25T16:36:26.204435+0000","last_active":"2024-04-25T16:37:16.206320+0000","last_peered":"2024-04-25T16:37:16.206320+0000","last_clean":"2024-04-25T16:37:16.206320+0000","last_became_active":"2024-04-25T16:36:26.203592+0000","last_became_peered":"2024-04-25T16:36:26.203592+0000","last_unstale":"2024-04-25T16:37:16.206320+0000","last_undegraded":"2024-04-25T16:37:16.206320+0000","last_fullsized":"2024-04-25T16:37:16.206320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:18.623888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211261+0000","last_change":"2024-04-25T16:36:26.971531+0000","last_active":"2024-04-25T16:37:16.211261+0000","last_peered":"2024-04-25T16:37:16.211261+0000","last_clean":"2024-04-25T16:37:16.211261+0000","last_became_active":"2024-04-25T16:36:26.969904+0000","last_became_peered":"2024-04-25T16:36:26.969904+0000","last_unstale":"2024-04-25T16:37:16.211261+0000","last_undegraded":"2024-04-25T16:37:16.211261+0000","last_fullsized":"2024-04-25T16:37:16.211261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:47.468692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211661+0000","last_change":"2024-04-25T16:36:26.981281+0000","last_active":"2024-04-25T16:37:16.211661+0000","last_peered":"2024-04-25T16:37:16.211661+0000","last_clean":"2024-04-25T16:37:16.211661+0000","last_became_active":"2024-04-25T16:36:26.980942+0000","last_became_peered":"2024-04-25T16:36:26.980942+0000","last_unstale":"2024-04-25T16:37:16.211661+0000","last_undegraded":"2024-04-25T16:37:16.211661+0000","last_fullsized":"2024-04-25T16:37:16.211661+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:30.713519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.208909+0000","last_change":"2024-04-25T16:36:26.989589+0000","last_active":"2024-04-25T16:37:16.208909+0000","last_peered":"2024-04-25T16:37:16.208909+0000","last_clean":"2024-04-25T16:37:16.208909+0000","last_became_active":"2024-04-25T16:36:26.989127+0000","last_became_peered":"2024-04-25T16:36:26.989127+0000","last_unstale":"2024-04-25T16:37:16.208909+0000","last_undegraded":"2024-04-25T16:37:16.208909+0000","last_fullsized":"2024-04-25T16:37:16.208909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:53.989995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.208978+0000","last_change":"2024-04-25T16:36:26.983784+0000","last_active":"2024-04-25T16:37:16.208978+0000","last_peered":"2024-04-25T16:37:16.208978+0000","last_clean":"2024-04-25T16:37:16.208978+0000","last_became_active":"2024-04-25T16:36:26.983256+0000","last_became_peered":"2024-04-25T16:36:26.983256+0000","last_unstale":"2024-04-25T16:37:16.208978+0000","last_undegraded":"2024-04-25T16:37:16.208978+0000","last_fullsized":"2024-04-25T16:37:16.208978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:44.469717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207366+0000","last_change":"2024-04-25T16:36:26.986223+0000","last_active":"2024-04-25T16:37:16.207366+0000","last_peered":"2024-04-25T16:37:16.207366+0000","last_clean":"2024-04-25T16:37:16.207366+0000","last_became_active":"2024-04-25T16:36:26.985511+0000","last_became_peered":"2024-04-25T16:36:26.985511+0000","last_unstale":"2024-04-25T16:37:16.207366+0000","last_undegraded":"2024-04-25T16:37:16.207366+0000","last_fullsized":"2024-04-25T16:37:16.207366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:46.269298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217428+0000","last_change":"2024-04-25T16:37:16.217428+0000","last_active":"2024-04-25T16:37:16.217428+0000","last_peered":"2024-04-25T16:37:16.217428+0000","last_clean":"2024-04-25T16:37:16.217428+0000","last_became_active":"2024-04-25T16:37:16.216975+0000","last_became_peered":"2024-04-25T16:37:16.216975+0000","last_unstale":"2024-04-25T16:37:16.217428+0000","last_undegraded":"2024-04-25T16:37:16.217428+0000","last_fullsized":"2024-04-25T16:37:16.217428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:56.498477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.212492+0000","last_change":"2024-04-25T16:36:26.974556+0000","last_active":"2024-04-25T16:37:16.212492+0000","last_peered":"2024-04-25T16:37:16.212492+0000","last_clean":"2024-04-25T16:37:16.212492+0000","last_became_active":"2024-04-25T16:36:26.974179+0000","last_became_peered":"2024-04-25T16:36:26.974179+0000","last_unstale":"2024-04-25T16:37:16.212492+0000","last_undegraded":"2024-04-25T16:37:16.212492+0000","last_fullsized":"2024-04-25T16:37:16.212492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:35.313778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.209638+0000","last_change":"2024-04-25T16:36:26.985614+0000","last_active":"2024-04-25T16:37:16.209638+0000","last_peered":"2024-04-25T16:37:16.209638+0000","last_clean":"2024-04-25T16:37:16.209638+0000","last_became_active":"2024-04-25T16:36:26.985259+0000","last_became_peered":"2024-04-25T16:36:26.985259+0000","last_unstale":"2024-04-25T16:37:16.209638+0000","last_undegraded":"2024-04-25T16:37:16.209638+0000","last_fullsized":"2024-04-25T16:37:16.209638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.850086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366545+0000","last_change":"2024-04-25T16:36:26.981898+0000","last_active":"2024-04-25T16:37:16.366545+0000","last_peered":"2024-04-25T16:37:16.366545+0000","last_clean":"2024-04-25T16:37:16.366545+0000","last_became_active":"2024-04-25T16:36:26.981554+0000","last_became_peered":"2024-04-25T16:36:26.981554+0000","last_unstale":"2024-04-25T16:37:16.366545+0000","last_undegraded":"2024-04-25T16:37:16.366545+0000","last_fullsized":"2024-04-25T16:37:16.366545+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:20.836846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.203968+0000","last_change":"2024-04-25T16:36:26.970983+0000","last_active":"2024-04-25T16:37:16.203968+0000","last_peered":"2024-04-25T16:37:16.203968+0000","last_clean":"2024-04-25T16:37:16.203968+0000","last_became_active":"2024-04-25T16:36:26.970644+0000","last_became_peered":"2024-04-25T16:36:26.970644+0000","last_unstale":"2024-04-25T16:37:16.203968+0000","last_undegraded":"2024-04-25T16:37:16.203968+0000","last_fullsized":"2024-04-25T16:37:16.203968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:33.285236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366489+0000","last_change":"2024-04-25T16:36:26.982347+0000","last_active":"2024-04-25T16:37:16.366489+0000","last_peered":"2024-04-25T16:37:16.366489+0000","last_clean":"2024-04-25T16:37:16.366489+0000","last_became_active":"2024-04-25T16:36:26.981970+0000","last_became_peered":"2024-04-25T16:36:26.981970+0000","last_unstale":"2024-04-25T16:37:16.366489+0000","last_undegraded":"2024-04-25T16:37:16.366489+0000","last_fullsized":"2024-04-25T16:37:16.366489+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:43.589483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206332+0000","last_change":"2024-04-25T16:36:26.975329+0000","last_active":"2024-04-25T16:37:16.206332+0000","last_peered":"2024-04-25T16:37:16.206332+0000","last_clean":"2024-04-25T16:37:16.206332+0000","last_became_active":"2024-04-25T16:36:26.974791+0000","last_became_peered":"2024-04-25T16:36:26.974791+0000","last_unstale":"2024-04-25T16:37:16.206332+0000","last_undegraded":"2024-04-25T16:37:16.206332+0000","last_fullsized":"2024-04-25T16:37:16.206332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:43.759074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206186+0000","last_change":"2024-04-25T16:36:26.983841+0000","last_active":"2024-04-25T16:37:16.206186+0000","last_peered":"2024-04-25T16:37:16.206186+0000","last_clean":"2024-04-25T16:37:16.206186+0000","last_became_active":"2024-04-25T16:36:26.982595+0000","last_became_peered":"2024-04-25T16:36:26.982595+0000","last_unstale":"2024-04-25T16:37:16.206186+0000","last_undegraded":"2024-04-25T16:37:16.206186+0000","last_fullsized":"2024-04-25T16:37:16.206186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.793016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211256+0000","last_change":"2024-04-25T16:36:26.984713+0000","last_active":"2024-04-25T16:37:16.211256+0000","last_peered":"2024-04-25T16:37:16.211256+0000","last_clean":"2024-04-25T16:37:16.211256+0000","last_became_active":"2024-04-25T16:36:26.984330+0000","last_became_peered":"2024-04-25T16:36:26.984330+0000","last_unstale":"2024-04-25T16:37:16.211256+0000","last_undegraded":"2024-04-25T16:37:16.211256+0000","last_fullsized":"2024-04-25T16:37:16.211256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:33.028204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246733,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":442,"num_read_kb":285,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":195141,"data_compressed":5601,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710749,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":831488,"data_stored":401650,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906978,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1054155,"data_compressed":43355,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":401864,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266732,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":839680,"data_stored":401762,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":993714,"data_compressed":41488,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":831488,"data_stored":401253,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":462092,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":402524,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":463833,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":847872,"data_stored":401564,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":401690,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822717,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":993398,"data_compressed":41488,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62624,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:37:45.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T16:37:45.365 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:37:45.741 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:45 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:37:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:45 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:37:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:45.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:37:45.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:37:45.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:37:45.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:45 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' 2024-04-25T16:37:46.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:46 smithi112 ceph-mon[29122]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T16:37:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:46 smithi133 ceph-mon[32691]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T16:37:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:46 smithi116 ceph-mon[32712]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T16:37:47.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 -- 172.21.15.112:0/4000762163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a41031e0 msgr2=0x7f81a410f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:47.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 --2- 172.21.15.112:0/4000762163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a41031e0 0x7f81a410f760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f819c00b0a0 tx=0x7f819c00b3b0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 -- 172.21.15.112:0/4000762163 shutdown_connections 2024-04-25T16:37:47.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 --2- 172.21.15.112:0/4000762163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a41031e0 0x7f81a410f760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 --2- 172.21.15.112:0/4000762163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a41088c0 0x7f81a4108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 --2- 172.21.15.112:0/4000762163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a4102890 0x7f81a4102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 -- 172.21.15.112:0/4000762163 >> 172.21.15.112:0/4000762163 conn(0x7f81a40fe3f0 msgr2=0x7f81a4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:47.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 -- 172.21.15.112:0/4000762163 shutdown_connections 2024-04-25T16:37:47.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.088+0000 7f81aae0d700 1 -- 172.21.15.112:0/4000762163 wait complete. 2024-04-25T16:37:47.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 Processor -- start 2024-04-25T16:37:47.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 -- start start 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 0x7f81a41a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a41031e0 0x7f81a41a0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a41088c0 0x7f81a419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41121e0 con 0x7f81a4102890 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4112060 con 0x7f81a41088c0 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81aae0d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4112360 con 0x7f81a41031e0 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81a8ba9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 0x7f81a41a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81a8ba9700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 0x7f81a41a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48972/0 (socket says 172.21.15.112:48972) 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.089+0000 7f81a8ba9700 1 -- 172.21.15.112:0/1954372880 learned_addr learned my addr 172.21.15.112:0/1954372880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a3fff700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a41031e0 0x7f81a41a0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:47.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a93aa700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a41088c0 0x7f81a419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a8ba9700 1 -- 172.21.15.112:0/1954372880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a41031e0 msgr2=0x7f81a41a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a8ba9700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a41031e0 0x7f81a41a0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a8ba9700 1 -- 172.21.15.112:0/1954372880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a41088c0 msgr2=0x7f81a419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a8ba9700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a41088c0 0x7f81a419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a8ba9700 1 -- 172.21.15.112:0/1954372880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a419b280 con 0x7f81a4102890 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a8ba9700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 0x7f81a41a09c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f819800dbb0 tx=0x7f819800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198007d70 con 0x7f81a4102890 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81980044d0 con 0x7f81a4102890 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a419b570 con 0x7f81a4102890 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198005260 con 0x7f81a4102890 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.090+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41a7770 con 0x7f81a4102890 2024-04-25T16:37:47.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.091+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a4103fd0 con 0x7f81a4102890 2024-04-25T16:37:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.093+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f81980054c0 con 0x7f81a4102890 2024-04-25T16:37:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.093+0000 7f81a1ffb700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f81940afb80 0x7f81940b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.094+0000 7f81a3fff700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f81940afb80 0x7f81940b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.094+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f81980dec00 con 0x7f81a4102890 2024-04-25T16:37:47.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.094+0000 7f81a3fff700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f81940afb80 0x7f81940b2090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8190006fd0 tx=0x7f8190008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:47.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.097+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8198050ff0 con 0x7f81a4102890 2024-04-25T16:37:47.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.290+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81a419bff0 con 0x7f81940afb80 2024-04-25T16:37:47.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.298+0000 7f81a1ffb700 1 -- 172.21.15.112:0/1954372880 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259043 (secure 0 0 0) 0x7f81a419bff0 con 0x7f81940afb80 2024-04-25T16:37:47.300 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f81940afb80 msgr2=0x7f81940b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f81940afb80 0x7f81940b2090 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8190006fd0 tx=0x7f8190008040 comp rx=0 tx=0).stop 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 msgr2=0x7f81a41a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 0x7f81a41a09c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f819800dbb0 tx=0x7f819800df70 comp rx=0 tx=0).stop 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 shutdown_connections 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f81940afb80 0x7f81940b2090 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81a41088c0 0x7f81a419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f81a4102890 0x7f81a41a09c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 --2- 172.21.15.112:0/1954372880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81a41031e0 0x7f81a41a0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 >> 172.21.15.112:0/1954372880 conn(0x7f81a40fe3f0 msgr2=0x7f81a4106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:47.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 shutdown_connections 2024-04-25T16:37:47.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:47.301+0000 7f81aae0d700 1 -- 172.21.15.112:0/1954372880 wait complete. 2024-04-25T16:37:47.303 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T16:37:47.672 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:47 smithi112 ceph-mon[29122]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:37:47.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:47 smithi133 ceph-mon[32691]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:37:47.984 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-04-25T16:37:47.101371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655649,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":543,"num_read_kb":369,"num_write":281,"num_write_kb":2260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":2951862,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":396,"ondisk_log_size":396,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":346964,"kb_used_data":10212,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124594860,"statfs":{"total":1151940427776,"available":1151585136640,"internally_reserved":0,"allocated":10457088,"data_stored":6779499,"data_compressed":182058,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":344634683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003302"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.097646+0000","last_change":"2024-04-25T16:36:26.988707+0000","last_active":"2024-04-25T16:37:45.097646+0000","last_peered":"2024-04-25T16:37:45.097646+0000","last_clean":"2024-04-25T16:37:45.097646+0000","last_became_active":"2024-04-25T16:36:26.988302+0000","last_became_peered":"2024-04-25T16:36:26.988302+0000","last_unstale":"2024-04-25T16:37:45.097646+0000","last_undegraded":"2024-04-25T16:37:45.097646+0000","last_fullsized":"2024-04-25T16:37:45.097646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:47:44.074090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207541+0000","last_change":"2024-04-25T16:36:26.988147+0000","last_active":"2024-04-25T16:37:16.207541+0000","last_peered":"2024-04-25T16:37:16.207541+0000","last_clean":"2024-04-25T16:37:16.207541+0000","last_became_active":"2024-04-25T16:36:26.987695+0000","last_became_peered":"2024-04-25T16:36:26.987695+0000","last_unstale":"2024-04-25T16:37:16.207541+0000","last_undegraded":"2024-04-25T16:37:16.207541+0000","last_fullsized":"2024-04-25T16:37:16.207541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:09.330235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207145+0000","last_change":"2024-04-25T16:36:26.986601+0000","last_active":"2024-04-25T16:37:16.207145+0000","last_peered":"2024-04-25T16:37:16.207145+0000","last_clean":"2024-04-25T16:37:16.207145+0000","last_became_active":"2024-04-25T16:36:26.985431+0000","last_became_peered":"2024-04-25T16:36:26.985431+0000","last_unstale":"2024-04-25T16:37:16.207145+0000","last_undegraded":"2024-04-25T16:37:16.207145+0000","last_fullsized":"2024-04-25T16:37:16.207145+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:09:52.756550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204375+0000","last_change":"2024-04-25T16:36:26.985007+0000","last_active":"2024-04-25T16:37:16.204375+0000","last_peered":"2024-04-25T16:37:16.204375+0000","last_clean":"2024-04-25T16:37:16.204375+0000","last_became_active":"2024-04-25T16:36:26.983000+0000","last_became_peered":"2024-04-25T16:36:26.983000+0000","last_unstale":"2024-04-25T16:37:16.204375+0000","last_undegraded":"2024-04-25T16:37:16.204375+0000","last_fullsized":"2024-04-25T16:37:16.204375+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:02.324409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.080259+0000","last_change":"2024-04-25T16:36:26.970714+0000","last_active":"2024-04-25T16:37:45.080259+0000","last_peered":"2024-04-25T16:37:45.080259+0000","last_clean":"2024-04-25T16:37:45.080259+0000","last_became_active":"2024-04-25T16:36:26.970323+0000","last_became_peered":"2024-04-25T16:36:26.970323+0000","last_unstale":"2024-04-25T16:37:45.080259+0000","last_undegraded":"2024-04-25T16:37:45.080259+0000","last_fullsized":"2024-04-25T16:37:45.080259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:41.113178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314698+0000","last_change":"2024-04-25T16:36:26.988388+0000","last_active":"2024-04-25T16:37:16.314698+0000","last_peered":"2024-04-25T16:37:16.314698+0000","last_clean":"2024-04-25T16:37:16.314698+0000","last_became_active":"2024-04-25T16:36:26.988090+0000","last_became_peered":"2024-04-25T16:36:26.988090+0000","last_unstale":"2024-04-25T16:37:16.314698+0000","last_undegraded":"2024-04-25T16:37:16.314698+0000","last_fullsized":"2024-04-25T16:37:16.314698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:02:19.666488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210381+0000","last_change":"2024-04-25T16:36:26.971244+0000","last_active":"2024-04-25T16:37:16.210381+0000","last_peered":"2024-04-25T16:37:16.210381+0000","last_clean":"2024-04-25T16:37:16.210381+0000","last_became_active":"2024-04-25T16:36:26.970900+0000","last_became_peered":"2024-04-25T16:36:26.970900+0000","last_unstale":"2024-04-25T16:37:16.210381+0000","last_undegraded":"2024-04-25T16:37:16.210381+0000","last_fullsized":"2024-04-25T16:37:16.210381+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:46.434230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.096451+0000","last_change":"2024-04-25T16:36:26.982614+0000","last_active":"2024-04-25T16:37:45.096451+0000","last_peered":"2024-04-25T16:37:45.096451+0000","last_clean":"2024-04-25T16:37:45.096451+0000","last_became_active":"2024-04-25T16:36:26.981911+0000","last_became_peered":"2024-04-25T16:36:26.981911+0000","last_unstale":"2024-04-25T16:37:45.096451+0000","last_undegraded":"2024-04-25T16:37:45.096451+0000","last_fullsized":"2024-04-25T16:37:45.096451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:24:36.186614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.212825+0000","last_change":"2024-04-25T16:37:16.212825+0000","last_active":"2024-04-25T16:37:16.212825+0000","last_peered":"2024-04-25T16:37:16.212825+0000","last_clean":"2024-04-25T16:37:16.212825+0000","last_became_active":"2024-04-25T16:37:16.212387+0000","last_became_peered":"2024-04-25T16:37:16.212387+0000","last_unstale":"2024-04-25T16:37:16.212825+0000","last_undegraded":"2024-04-25T16:37:16.212825+0000","last_fullsized":"2024-04-25T16:37:16.212825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:49.060551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.315082+0000","last_change":"2024-04-25T16:36:26.989299+0000","last_active":"2024-04-25T16:37:16.315082+0000","last_peered":"2024-04-25T16:37:16.315082+0000","last_clean":"2024-04-25T16:37:16.315082+0000","last_became_active":"2024-04-25T16:36:26.988826+0000","last_became_peered":"2024-04-25T16:36:26.988826+0000","last_unstale":"2024-04-25T16:37:16.315082+0000","last_undegraded":"2024-04-25T16:37:16.315082+0000","last_fullsized":"2024-04-25T16:37:16.315082+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:40.549866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211051+0000","last_change":"2024-04-25T16:36:26.971847+0000","last_active":"2024-04-25T16:37:16.211051+0000","last_peered":"2024-04-25T16:37:16.211051+0000","last_clean":"2024-04-25T16:37:16.211051+0000","last_became_active":"2024-04-25T16:36:26.971417+0000","last_became_peered":"2024-04-25T16:36:26.971417+0000","last_unstale":"2024-04-25T16:37:16.211051+0000","last_undegraded":"2024-04-25T16:37:16.211051+0000","last_fullsized":"2024-04-25T16:37:16.211051+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:55.075777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204499+0000","last_change":"2024-04-25T16:36:26.985714+0000","last_active":"2024-04-25T16:37:16.204499+0000","last_peered":"2024-04-25T16:37:16.204499+0000","last_clean":"2024-04-25T16:37:16.204499+0000","last_became_active":"2024-04-25T16:36:26.985402+0000","last_became_peered":"2024-04-25T16:36:26.985402+0000","last_unstale":"2024-04-25T16:37:16.204499+0000","last_undegraded":"2024-04-25T16:37:16.204499+0000","last_fullsized":"2024-04-25T16:37:16.204499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:32.953895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206083+0000","last_change":"2024-04-25T16:36:26.976917+0000","last_active":"2024-04-25T16:37:16.206083+0000","last_peered":"2024-04-25T16:37:16.206083+0000","last_clean":"2024-04-25T16:37:16.206083+0000","last_became_active":"2024-04-25T16:36:26.976452+0000","last_became_peered":"2024-04-25T16:36:26.976452+0000","last_unstale":"2024-04-25T16:37:16.206083+0000","last_undegraded":"2024-04-25T16:37:16.206083+0000","last_fullsized":"2024-04-25T16:37:16.206083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:45.530249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207063+0000","last_change":"2024-04-25T16:36:26.987433+0000","last_active":"2024-04-25T16:37:16.207063+0000","last_peered":"2024-04-25T16:37:16.207063+0000","last_clean":"2024-04-25T16:37:16.207063+0000","last_became_active":"2024-04-25T16:36:26.987106+0000","last_became_peered":"2024-04-25T16:36:26.987106+0000","last_unstale":"2024-04-25T16:37:16.207063+0000","last_undegraded":"2024-04-25T16:37:16.207063+0000","last_fullsized":"2024-04-25T16:37:16.207063+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:33.436034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206750+0000","last_change":"2024-04-25T16:36:26.984530+0000","last_active":"2024-04-25T16:37:16.206750+0000","last_peered":"2024-04-25T16:37:16.206750+0000","last_clean":"2024-04-25T16:37:16.206750+0000","last_became_active":"2024-04-25T16:36:26.984220+0000","last_became_peered":"2024-04-25T16:36:26.984220+0000","last_unstale":"2024-04-25T16:37:16.206750+0000","last_undegraded":"2024-04-25T16:37:16.206750+0000","last_fullsized":"2024-04-25T16:37:16.206750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:13:06.345465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206097+0000","last_change":"2024-04-25T16:36:26.980249+0000","last_active":"2024-04-25T16:37:16.206097+0000","last_peered":"2024-04-25T16:37:16.206097+0000","last_clean":"2024-04-25T16:37:16.206097+0000","last_became_active":"2024-04-25T16:36:26.979861+0000","last_became_peered":"2024-04-25T16:36:26.979861+0000","last_unstale":"2024-04-25T16:37:16.206097+0000","last_undegraded":"2024-04-25T16:37:16.206097+0000","last_fullsized":"2024-04-25T16:37:16.206097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:26.753789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.105992+0000","last_change":"2024-04-25T16:36:26.988022+0000","last_active":"2024-04-25T16:37:45.105992+0000","last_peered":"2024-04-25T16:37:45.105992+0000","last_clean":"2024-04-25T16:37:45.105992+0000","last_became_active":"2024-04-25T16:36:26.987609+0000","last_became_peered":"2024-04-25T16:36:26.987609+0000","last_unstale":"2024-04-25T16:37:45.105992+0000","last_undegraded":"2024-04-25T16:37:45.105992+0000","last_fullsized":"2024-04-25T16:37:45.105992+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:45.819517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214455+0000","last_change":"2024-04-25T16:36:26.978511+0000","last_active":"2024-04-25T16:37:16.214455+0000","last_peered":"2024-04-25T16:37:16.214455+0000","last_clean":"2024-04-25T16:37:16.214455+0000","last_became_active":"2024-04-25T16:36:26.978180+0000","last_became_peered":"2024-04-25T16:36:26.978180+0000","last_unstale":"2024-04-25T16:37:16.214455+0000","last_undegraded":"2024-04-25T16:37:16.214455+0000","last_fullsized":"2024-04-25T16:37:16.214455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:58.931649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.085741+0000","last_change":"2024-04-25T16:37:16.217509+0000","last_active":"2024-04-25T16:37:45.085741+0000","last_peered":"2024-04-25T16:37:45.085741+0000","last_clean":"2024-04-25T16:37:45.085741+0000","last_became_active":"2024-04-25T16:37:16.217018+0000","last_became_peered":"2024-04-25T16:37:16.217018+0000","last_unstale":"2024-04-25T16:37:45.085741+0000","last_undegraded":"2024-04-25T16:37:45.085741+0000","last_fullsized":"2024-04-25T16:37:45.085741+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:43.703130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"75'9","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214254+0000","last_change":"2024-04-25T16:36:26.216356+0000","last_active":"2024-04-25T16:37:16.214254+0000","last_peered":"2024-04-25T16:37:16.214254+0000","last_clean":"2024-04-25T16:37:16.214254+0000","last_became_active":"2024-04-25T16:36:26.215935+0000","last_became_peered":"2024-04-25T16:36:26.215935+0000","last_unstale":"2024-04-25T16:37:16.214254+0000","last_undegraded":"2024-04-25T16:37:16.214254+0000","last_fullsized":"2024-04-25T16:37:16.214254+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:58.186003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.093473+0000","last_change":"2024-04-25T16:36:26.981307+0000","last_active":"2024-04-25T16:37:45.093473+0000","last_peered":"2024-04-25T16:37:45.093473+0000","last_clean":"2024-04-25T16:37:45.093473+0000","last_became_active":"2024-04-25T16:36:26.980973+0000","last_became_peered":"2024-04-25T16:36:26.980973+0000","last_unstale":"2024-04-25T16:37:45.093473+0000","last_undegraded":"2024-04-25T16:37:45.093473+0000","last_fullsized":"2024-04-25T16:37:45.093473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:45.469266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366820+0000","last_change":"2024-04-25T16:36:26.218839+0000","last_active":"2024-04-25T16:37:16.366820+0000","last_peered":"2024-04-25T16:37:16.366820+0000","last_clean":"2024-04-25T16:37:16.366820+0000","last_became_active":"2024-04-25T16:36:26.218487+0000","last_became_peered":"2024-04-25T16:36:26.218487+0000","last_unstale":"2024-04-25T16:37:16.366820+0000","last_undegraded":"2024-04-25T16:37:16.366820+0000","last_fullsized":"2024-04-25T16:37:16.366820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:05.164084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207745+0000","last_change":"2024-04-25T16:36:26.977277+0000","last_active":"2024-04-25T16:37:16.207745+0000","last_peered":"2024-04-25T16:37:16.207745+0000","last_clean":"2024-04-25T16:37:16.207745+0000","last_became_active":"2024-04-25T16:36:26.974805+0000","last_became_peered":"2024-04-25T16:36:26.974805+0000","last_unstale":"2024-04-25T16:37:16.207745+0000","last_undegraded":"2024-04-25T16:37:16.207745+0000","last_fullsized":"2024-04-25T16:37:16.207745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:45.389875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.208108+0000","last_change":"2024-04-25T16:36:26.215757+0000","last_active":"2024-04-25T16:37:16.208108+0000","last_peered":"2024-04-25T16:37:16.208108+0000","last_clean":"2024-04-25T16:37:16.208108+0000","last_became_active":"2024-04-25T16:36:26.215424+0000","last_became_peered":"2024-04-25T16:36:26.215424+0000","last_unstale":"2024-04-25T16:37:16.208108+0000","last_undegraded":"2024-04-25T16:37:16.208108+0000","last_fullsized":"2024-04-25T16:37:16.208108+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:23.094599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206512+0000","last_change":"2024-04-25T16:36:26.978552+0000","last_active":"2024-04-25T16:37:16.206512+0000","last_peered":"2024-04-25T16:37:16.206512+0000","last_clean":"2024-04-25T16:37:16.206512+0000","last_became_active":"2024-04-25T16:36:26.978185+0000","last_became_peered":"2024-04-25T16:36:26.978185+0000","last_unstale":"2024-04-25T16:37:16.206512+0000","last_undegraded":"2024-04-25T16:37:16.206512+0000","last_fullsized":"2024-04-25T16:37:16.206512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:19.493267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207606+0000","last_change":"2024-04-25T16:36:26.207361+0000","last_active":"2024-04-25T16:37:16.207606+0000","last_peered":"2024-04-25T16:37:16.207606+0000","last_clean":"2024-04-25T16:37:16.207606+0000","last_became_active":"2024-04-25T16:36:26.206968+0000","last_became_peered":"2024-04-25T16:36:26.206968+0000","last_unstale":"2024-04-25T16:37:16.207606+0000","last_undegraded":"2024-04-25T16:37:16.207606+0000","last_fullsized":"2024-04-25T16:37:16.207606+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:23:54.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204182+0000","last_change":"2024-04-25T16:36:26.979723+0000","last_active":"2024-04-25T16:37:16.204182+0000","last_peered":"2024-04-25T16:37:16.204182+0000","last_clean":"2024-04-25T16:37:16.204182+0000","last_became_active":"2024-04-25T16:36:26.979377+0000","last_became_peered":"2024-04-25T16:36:26.979377+0000","last_unstale":"2024-04-25T16:37:16.204182+0000","last_undegraded":"2024-04-25T16:37:16.204182+0000","last_fullsized":"2024-04-25T16:37:16.204182+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:50.312759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":100,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:41.351181+0000","last_change":"2024-04-25T16:36:26.217074+0000","last_active":"2024-04-25T16:37:41.351181+0000","last_peered":"2024-04-25T16:37:41.351181+0000","last_clean":"2024-04-25T16:37:41.351181+0000","last_became_active":"2024-04-25T16:36:26.216686+0000","last_became_peered":"2024-04-25T16:36:26.216686+0000","last_unstale":"2024-04-25T16:37:41.351181+0000","last_undegraded":"2024-04-25T16:37:41.351181+0000","last_fullsized":"2024-04-25T16:37:41.351181+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:35.780179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.100712+0000","last_change":"2024-04-25T16:37:16.216828+0000","last_active":"2024-04-25T16:37:45.100712+0000","last_peered":"2024-04-25T16:37:45.100712+0000","last_clean":"2024-04-25T16:37:45.100712+0000","last_became_active":"2024-04-25T16:37:16.216350+0000","last_became_peered":"2024-04-25T16:37:16.216350+0000","last_unstale":"2024-04-25T16:37:45.100712+0000","last_undegraded":"2024-04-25T16:37:45.100712+0000","last_fullsized":"2024-04-25T16:37:45.100712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:32.705434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314600+0000","last_change":"2024-04-25T16:36:26.211703+0000","last_active":"2024-04-25T16:37:16.314600+0000","last_peered":"2024-04-25T16:37:16.314600+0000","last_clean":"2024-04-25T16:37:16.314600+0000","last_became_active":"2024-04-25T16:36:26.211322+0000","last_became_peered":"2024-04-25T16:36:26.211322+0000","last_unstale":"2024-04-25T16:37:16.314600+0000","last_undegraded":"2024-04-25T16:37:16.314600+0000","last_fullsized":"2024-04-25T16:37:16.314600+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:15.353208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210437+0000","last_change":"2024-04-25T16:37:16.210437+0000","last_active":"2024-04-25T16:37:16.210437+0000","last_peered":"2024-04-25T16:37:16.210437+0000","last_clean":"2024-04-25T16:37:16.210437+0000","last_became_active":"2024-04-25T16:37:16.210030+0000","last_became_peered":"2024-04-25T16:37:16.210030+0000","last_unstale":"2024-04-25T16:37:16.210437+0000","last_undegraded":"2024-04-25T16:37:16.210437+0000","last_fullsized":"2024-04-25T16:37:16.210437+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.511125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,8],"acting":[5,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206083+0000","last_change":"2024-04-25T16:36:26.217225+0000","last_active":"2024-04-25T16:37:16.206083+0000","last_peered":"2024-04-25T16:37:16.206083+0000","last_clean":"2024-04-25T16:37:16.206083+0000","last_became_active":"2024-04-25T16:36:26.216777+0000","last_became_peered":"2024-04-25T16:36:26.216777+0000","last_unstale":"2024-04-25T16:37:16.206083+0000","last_undegraded":"2024-04-25T16:37:16.206083+0000","last_fullsized":"2024-04-25T16:37:16.206083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:21.433977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.089652+0000","last_change":"2024-04-25T16:36:26.970197+0000","last_active":"2024-04-25T16:37:45.089652+0000","last_peered":"2024-04-25T16:37:45.089652+0000","last_clean":"2024-04-25T16:37:45.089652+0000","last_became_active":"2024-04-25T16:36:26.969844+0000","last_became_peered":"2024-04-25T16:36:26.969844+0000","last_unstale":"2024-04-25T16:37:45.089652+0000","last_undegraded":"2024-04-25T16:37:45.089652+0000","last_fullsized":"2024-04-25T16:37:45.089652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:35.397143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.203900+0000","last_change":"2024-04-25T16:36:26.214731+0000","last_active":"2024-04-25T16:37:16.203900+0000","last_peered":"2024-04-25T16:37:16.203900+0000","last_clean":"2024-04-25T16:37:16.203900+0000","last_became_active":"2024-04-25T16:36:26.214343+0000","last_became_peered":"2024-04-25T16:36:26.214343+0000","last_unstale":"2024-04-25T16:37:16.203900+0000","last_undegraded":"2024-04-25T16:37:16.203900+0000","last_fullsized":"2024-04-25T16:37:16.203900+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:08:10.156665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214350+0000","last_change":"2024-04-25T16:36:26.981458+0000","last_active":"2024-04-25T16:37:16.214350+0000","last_peered":"2024-04-25T16:37:16.214350+0000","last_clean":"2024-04-25T16:37:16.214350+0000","last_became_active":"2024-04-25T16:36:26.981116+0000","last_became_peered":"2024-04-25T16:36:26.981116+0000","last_unstale":"2024-04-25T16:37:16.214350+0000","last_undegraded":"2024-04-25T16:37:16.214350+0000","last_fullsized":"2024-04-25T16:37:16.214350+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:20.549670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'3","reported_seq":95,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:42.351935+0000","last_change":"2024-04-25T16:36:26.218690+0000","last_active":"2024-04-25T16:37:42.351935+0000","last_peered":"2024-04-25T16:37:42.351935+0000","last_clean":"2024-04-25T16:37:42.351935+0000","last_became_active":"2024-04-25T16:36:26.218287+0000","last_became_peered":"2024-04-25T16:36:26.218287+0000","last_unstale":"2024-04-25T16:37:42.351935+0000","last_undegraded":"2024-04-25T16:37:42.351935+0000","last_fullsized":"2024-04-25T16:37:42.351935+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:05:42.394413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206905+0000","last_change":"2024-04-25T16:36:26.980356+0000","last_active":"2024-04-25T16:37:16.206905+0000","last_peered":"2024-04-25T16:37:16.206905+0000","last_clean":"2024-04-25T16:37:16.206905+0000","last_became_active":"2024-04-25T16:36:26.980005+0000","last_became_peered":"2024-04-25T16:36:26.980005+0000","last_unstale":"2024-04-25T16:37:16.206905+0000","last_undegraded":"2024-04-25T16:37:16.206905+0000","last_fullsized":"2024-04-25T16:37:16.206905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:27:37.263655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210781+0000","last_change":"2024-04-25T16:36:26.217818+0000","last_active":"2024-04-25T16:37:16.210781+0000","last_peered":"2024-04-25T16:37:16.210781+0000","last_clean":"2024-04-25T16:37:16.210781+0000","last_became_active":"2024-04-25T16:36:26.217317+0000","last_became_peered":"2024-04-25T16:36:26.217317+0000","last_unstale":"2024-04-25T16:37:16.210781+0000","last_undegraded":"2024-04-25T16:37:16.210781+0000","last_fullsized":"2024-04-25T16:37:16.210781+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:29.985735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210320+0000","last_change":"2024-04-25T16:37:16.210320+0000","last_active":"2024-04-25T16:37:16.210320+0000","last_peered":"2024-04-25T16:37:16.210320+0000","last_clean":"2024-04-25T16:37:16.210320+0000","last_became_active":"2024-04-25T16:37:16.209914+0000","last_became_peered":"2024-04-25T16:37:16.209914+0000","last_unstale":"2024-04-25T16:37:16.210320+0000","last_undegraded":"2024-04-25T16:37:16.210320+0000","last_fullsized":"2024-04-25T16:37:16.210320+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:33.673346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'4","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204381+0000","last_change":"2024-04-25T16:36:26.208756+0000","last_active":"2024-04-25T16:37:16.204381+0000","last_peered":"2024-04-25T16:37:16.204381+0000","last_clean":"2024-04-25T16:37:16.204381+0000","last_became_active":"2024-04-25T16:36:26.208384+0000","last_became_peered":"2024-04-25T16:36:26.208384+0000","last_unstale":"2024-04-25T16:37:16.204381+0000","last_undegraded":"2024-04-25T16:37:16.204381+0000","last_fullsized":"2024-04-25T16:37:16.204381+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.849402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206484+0000","last_change":"2024-04-25T16:36:26.981611+0000","last_active":"2024-04-25T16:37:16.206484+0000","last_peered":"2024-04-25T16:37:16.206484+0000","last_clean":"2024-04-25T16:37:16.206484+0000","last_became_active":"2024-04-25T16:36:26.981311+0000","last_became_peered":"2024-04-25T16:36:26.981311+0000","last_unstale":"2024-04-25T16:37:16.206484+0000","last_undegraded":"2024-04-25T16:37:16.206484+0000","last_fullsized":"2024-04-25T16:37:16.206484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:05.360035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'28","reported_seq":201,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:43.353620+0000","last_change":"2024-04-25T16:36:26.214215+0000","last_active":"2024-04-25T16:37:43.353620+0000","last_peered":"2024-04-25T16:37:43.353620+0000","last_clean":"2024-04-25T16:37:43.353620+0000","last_became_active":"2024-04-25T16:36:26.213848+0000","last_became_peered":"2024-04-25T16:36:26.213848+0000","last_unstale":"2024-04-25T16:37:43.353620+0000","last_undegraded":"2024-04-25T16:37:43.353620+0000","last_fullsized":"2024-04-25T16:37:43.353620+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:32.589646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":154,"num_read_kb":136,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.096541+0000","last_change":"2024-04-25T16:36:26.976087+0000","last_active":"2024-04-25T16:37:45.096541+0000","last_peered":"2024-04-25T16:37:45.096541+0000","last_clean":"2024-04-25T16:37:45.096541+0000","last_became_active":"2024-04-25T16:36:26.975726+0000","last_became_peered":"2024-04-25T16:36:26.975726+0000","last_unstale":"2024-04-25T16:37:45.096541+0000","last_undegraded":"2024-04-25T16:37:45.096541+0000","last_fullsized":"2024-04-25T16:37:45.096541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:30:02.668930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'6","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.208985+0000","last_change":"2024-04-25T16:36:26.208815+0000","last_active":"2024-04-25T16:37:16.208985+0000","last_peered":"2024-04-25T16:37:16.208985+0000","last_clean":"2024-04-25T16:37:16.208985+0000","last_became_active":"2024-04-25T16:36:26.208485+0000","last_became_peered":"2024-04-25T16:36:26.208485+0000","last_unstale":"2024-04-25T16:37:16.208985+0000","last_undegraded":"2024-04-25T16:37:16.208985+0000","last_fullsized":"2024-04-25T16:37:16.208985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:11.984900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.205746+0000","last_change":"2024-04-25T16:36:26.975567+0000","last_active":"2024-04-25T16:37:16.205746+0000","last_peered":"2024-04-25T16:37:16.205746+0000","last_clean":"2024-04-25T16:37:16.205746+0000","last_became_active":"2024-04-25T16:36:26.975212+0000","last_became_peered":"2024-04-25T16:36:26.975212+0000","last_unstale":"2024-04-25T16:37:16.205746+0000","last_undegraded":"2024-04-25T16:37:16.205746+0000","last_fullsized":"2024-04-25T16:37:16.205746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:34.778011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204089+0000","last_change":"2024-04-25T16:36:26.214765+0000","last_active":"2024-04-25T16:37:16.204089+0000","last_peered":"2024-04-25T16:37:16.204089+0000","last_clean":"2024-04-25T16:37:16.204089+0000","last_became_active":"2024-04-25T16:36:26.214378+0000","last_became_peered":"2024-04-25T16:36:26.214378+0000","last_unstale":"2024-04-25T16:37:16.204089+0000","last_undegraded":"2024-04-25T16:37:16.204089+0000","last_fullsized":"2024-04-25T16:37:16.204089+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:00.358074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210983+0000","last_change":"2024-04-25T16:36:26.978074+0000","last_active":"2024-04-25T16:37:16.210983+0000","last_peered":"2024-04-25T16:37:16.210983+0000","last_clean":"2024-04-25T16:37:16.210983+0000","last_became_active":"2024-04-25T16:36:26.977706+0000","last_became_peered":"2024-04-25T16:36:26.977706+0000","last_unstale":"2024-04-25T16:37:16.210983+0000","last_undegraded":"2024-04-25T16:37:16.210983+0000","last_fullsized":"2024-04-25T16:37:16.210983+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:05.718026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'3","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:41.351062+0000","last_change":"2024-04-25T16:36:26.211526+0000","last_active":"2024-04-25T16:37:41.351062+0000","last_peered":"2024-04-25T16:37:41.351062+0000","last_clean":"2024-04-25T16:37:41.351062+0000","last_became_active":"2024-04-25T16:36:26.211134+0000","last_became_peered":"2024-04-25T16:36:26.211134+0000","last_unstale":"2024-04-25T16:37:41.351062+0000","last_undegraded":"2024-04-25T16:37:41.351062+0000","last_fullsized":"2024-04-25T16:37:41.351062+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:25:39.743843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210536+0000","last_change":"2024-04-25T16:36:26.979766+0000","last_active":"2024-04-25T16:37:16.210536+0000","last_peered":"2024-04-25T16:37:16.210536+0000","last_clean":"2024-04-25T16:37:16.210536+0000","last_became_active":"2024-04-25T16:36:26.979460+0000","last_became_peered":"2024-04-25T16:36:26.979460+0000","last_unstale":"2024-04-25T16:37:16.210536+0000","last_undegraded":"2024-04-25T16:37:16.210536+0000","last_fullsized":"2024-04-25T16:37:16.210536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:43.957289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'15","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206765+0000","last_change":"2024-04-25T16:36:26.212112+0000","last_active":"2024-04-25T16:37:16.206765+0000","last_peered":"2024-04-25T16:37:16.206765+0000","last_clean":"2024-04-25T16:37:16.206765+0000","last_became_active":"2024-04-25T16:36:26.211746+0000","last_became_peered":"2024-04-25T16:36:26.211746+0000","last_unstale":"2024-04-25T16:37:16.206765+0000","last_undegraded":"2024-04-25T16:37:16.206765+0000","last_fullsized":"2024-04-25T16:37:16.206765+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:11:24.682525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.203953+0000","last_change":"2024-04-25T16:36:26.980275+0000","last_active":"2024-04-25T16:37:16.203953+0000","last_peered":"2024-04-25T16:37:16.203953+0000","last_clean":"2024-04-25T16:37:16.203953+0000","last_became_active":"2024-04-25T16:36:26.979912+0000","last_became_peered":"2024-04-25T16:36:26.979912+0000","last_unstale":"2024-04-25T16:37:16.203953+0000","last_undegraded":"2024-04-25T16:37:16.203953+0000","last_fullsized":"2024-04-25T16:37:16.203953+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:28.802041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207890+0000","last_change":"2024-04-25T16:36:26.205090+0000","last_active":"2024-04-25T16:37:16.207890+0000","last_peered":"2024-04-25T16:37:16.207890+0000","last_clean":"2024-04-25T16:37:16.207890+0000","last_became_active":"2024-04-25T16:36:26.204707+0000","last_became_peered":"2024-04-25T16:36:26.204707+0000","last_unstale":"2024-04-25T16:37:16.207890+0000","last_undegraded":"2024-04-25T16:37:16.207890+0000","last_fullsized":"2024-04-25T16:37:16.207890+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:09:13.479241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.097455+0000","last_change":"2024-04-25T16:36:26.985815+0000","last_active":"2024-04-25T16:37:45.097455+0000","last_peered":"2024-04-25T16:37:45.097455+0000","last_clean":"2024-04-25T16:37:45.097455+0000","last_became_active":"2024-04-25T16:36:26.985472+0000","last_became_peered":"2024-04-25T16:36:26.985472+0000","last_unstale":"2024-04-25T16:37:45.097455+0000","last_undegraded":"2024-04-25T16:37:45.097455+0000","last_fullsized":"2024-04-25T16:37:45.097455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:33.883655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214288+0000","last_change":"2024-04-25T16:36:26.207352+0000","last_active":"2024-04-25T16:37:16.214288+0000","last_peered":"2024-04-25T16:37:16.214288+0000","last_clean":"2024-04-25T16:37:16.214288+0000","last_became_active":"2024-04-25T16:36:26.206975+0000","last_became_peered":"2024-04-25T16:36:26.206975+0000","last_unstale":"2024-04-25T16:37:16.214288+0000","last_undegraded":"2024-04-25T16:37:16.214288+0000","last_fullsized":"2024-04-25T16:37:16.214288+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:03.085034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314805+0000","last_change":"2024-04-25T16:36:26.982401+0000","last_active":"2024-04-25T16:37:16.314805+0000","last_peered":"2024-04-25T16:37:16.314805+0000","last_clean":"2024-04-25T16:37:16.314805+0000","last_became_active":"2024-04-25T16:36:26.982070+0000","last_became_peered":"2024-04-25T16:36:26.982070+0000","last_unstale":"2024-04-25T16:37:16.314805+0000","last_undegraded":"2024-04-25T16:37:16.314805+0000","last_fullsized":"2024-04-25T16:37:16.314805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:35.824276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210632+0000","last_change":"2024-04-25T16:36:26.214759+0000","last_active":"2024-04-25T16:37:16.210632+0000","last_peered":"2024-04-25T16:37:16.210632+0000","last_clean":"2024-04-25T16:37:16.210632+0000","last_became_active":"2024-04-25T16:36:26.214334+0000","last_became_peered":"2024-04-25T16:36:26.214334+0000","last_unstale":"2024-04-25T16:37:16.210632+0000","last_undegraded":"2024-04-25T16:37:16.210632+0000","last_fullsized":"2024-04-25T16:37:16.210632+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:00.537377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.314680+0000","last_change":"2024-04-25T16:36:26.988313+0000","last_active":"2024-04-25T16:37:16.314680+0000","last_peered":"2024-04-25T16:37:16.314680+0000","last_clean":"2024-04-25T16:37:16.314680+0000","last_became_active":"2024-04-25T16:36:26.987979+0000","last_became_peered":"2024-04-25T16:36:26.987979+0000","last_unstale":"2024-04-25T16:37:16.314680+0000","last_undegraded":"2024-04-25T16:37:16.314680+0000","last_fullsized":"2024-04-25T16:37:16.314680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:40.737779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204052+0000","last_change":"2024-04-25T16:36:26.982005+0000","last_active":"2024-04-25T16:37:16.204052+0000","last_peered":"2024-04-25T16:37:16.204052+0000","last_clean":"2024-04-25T16:37:16.204052+0000","last_became_active":"2024-04-25T16:36:26.981619+0000","last_became_peered":"2024-04-25T16:36:26.981619+0000","last_unstale":"2024-04-25T16:37:16.204052+0000","last_undegraded":"2024-04-25T16:37:16.204052+0000","last_fullsized":"2024-04-25T16:37:16.204052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:49.555114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"75'4","reported_seq":96,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:42.351777+0000","last_change":"2024-04-25T16:36:26.207188+0000","last_active":"2024-04-25T16:37:42.351777+0000","last_peered":"2024-04-25T16:37:42.351777+0000","last_clean":"2024-04-25T16:37:42.351777+0000","last_became_active":"2024-04-25T16:36:26.206497+0000","last_became_peered":"2024-04-25T16:36:26.206497+0000","last_unstale":"2024-04-25T16:37:42.351777+0000","last_undegraded":"2024-04-25T16:37:42.351777+0000","last_fullsized":"2024-04-25T16:37:42.351777+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:45.213263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.112956+0000","last_change":"2024-04-25T16:36:26.980524+0000","last_active":"2024-04-25T16:37:45.112956+0000","last_peered":"2024-04-25T16:37:45.112956+0000","last_clean":"2024-04-25T16:37:45.112956+0000","last_became_active":"2024-04-25T16:36:26.980195+0000","last_became_peered":"2024-04-25T16:36:26.980195+0000","last_unstale":"2024-04-25T16:37:45.112956+0000","last_undegraded":"2024-04-25T16:37:45.112956+0000","last_fullsized":"2024-04-25T16:37:45.112956+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:25.623403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"75'3","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204773+0000","last_change":"2024-04-25T16:36:26.208423+0000","last_active":"2024-04-25T16:37:16.204773+0000","last_peered":"2024-04-25T16:37:16.204773+0000","last_clean":"2024-04-25T16:37:16.204773+0000","last_became_active":"2024-04-25T16:36:26.208045+0000","last_became_peered":"2024-04-25T16:36:26.208045+0000","last_unstale":"2024-04-25T16:37:16.204773+0000","last_undegraded":"2024-04-25T16:37:16.204773+0000","last_fullsized":"2024-04-25T16:37:16.204773+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:47.727762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210324+0000","last_change":"2024-04-25T16:35:10.871536+0000","last_active":"2024-04-25T16:37:16.210324+0000","last_peered":"2024-04-25T16:37:16.210324+0000","last_clean":"2024-04-25T16:37:16.210324+0000","last_became_active":"2024-04-25T16:35:10.866041+0000","last_became_peered":"2024-04-25T16:35:10.866041+0000","last_unstale":"2024-04-25T16:37:16.210324+0000","last_undegraded":"2024-04-25T16:37:16.210324+0000","last_fullsized":"2024-04-25T16:37:16.210324+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:31:41.248932+0000","last_clean_scrub_stamp":"2024-04-25T16:31:41.248932+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:45.476632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217426+0000","last_change":"2024-04-25T16:37:16.217426+0000","last_active":"2024-04-25T16:37:16.217426+0000","last_peered":"2024-04-25T16:37:16.217426+0000","last_clean":"2024-04-25T16:37:16.217426+0000","last_became_active":"2024-04-25T16:37:16.216994+0000","last_became_peered":"2024-04-25T16:37:16.216994+0000","last_unstale":"2024-04-25T16:37:16.217426+0000","last_undegraded":"2024-04-25T16:37:16.217426+0000","last_fullsized":"2024-04-25T16:37:16.217426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:36.822278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210299+0000","last_change":"2024-04-25T16:36:26.219000+0000","last_active":"2024-04-25T16:37:16.210299+0000","last_peered":"2024-04-25T16:37:16.210299+0000","last_clean":"2024-04-25T16:37:16.210299+0000","last_became_active":"2024-04-25T16:36:26.218599+0000","last_became_peered":"2024-04-25T16:36:26.218599+0000","last_unstale":"2024-04-25T16:37:16.210299+0000","last_undegraded":"2024-04-25T16:37:16.210299+0000","last_fullsized":"2024-04-25T16:37:16.210299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:51.186499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"75'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.220811+0000","last_change":"2024-04-25T16:37:16.220811+0000","last_active":"2024-04-25T16:37:16.220811+0000","last_peered":"2024-04-25T16:37:16.220811+0000","last_clean":"2024-04-25T16:37:16.220811+0000","last_became_active":"2024-04-25T16:37:16.217208+0000","last_became_peered":"2024-04-25T16:37:16.217208+0000","last_unstale":"2024-04-25T16:37:16.220811+0000","last_undegraded":"2024-04-25T16:37:16.220811+0000","last_fullsized":"2024-04-25T16:37:16.220811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:56.658270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206074+0000","last_change":"2024-04-25T16:36:26.981916+0000","last_active":"2024-04-25T16:37:16.206074+0000","last_peered":"2024-04-25T16:37:16.206074+0000","last_clean":"2024-04-25T16:37:16.206074+0000","last_became_active":"2024-04-25T16:36:26.978975+0000","last_became_peered":"2024-04-25T16:36:26.978975+0000","last_unstale":"2024-04-25T16:37:16.206074+0000","last_undegraded":"2024-04-25T16:37:16.206074+0000","last_fullsized":"2024-04-25T16:37:16.206074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:43.748231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'8","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.214513+0000","last_change":"2024-04-25T16:36:26.221399+0000","last_active":"2024-04-25T16:37:16.214513+0000","last_peered":"2024-04-25T16:37:16.214513+0000","last_clean":"2024-04-25T16:37:16.214513+0000","last_became_active":"2024-04-25T16:36:26.220986+0000","last_became_peered":"2024-04-25T16:36:26.220986+0000","last_unstale":"2024-04-25T16:37:16.214513+0000","last_undegraded":"2024-04-25T16:37:16.214513+0000","last_fullsized":"2024-04-25T16:37:16.214513+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:59.793148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.100496+0000","last_change":"2024-04-25T16:36:26.974784+0000","last_active":"2024-04-25T16:37:45.100496+0000","last_peered":"2024-04-25T16:37:45.100496+0000","last_clean":"2024-04-25T16:37:45.100496+0000","last_became_active":"2024-04-25T16:36:26.974439+0000","last_became_peered":"2024-04-25T16:36:26.974439+0000","last_unstale":"2024-04-25T16:37:45.100496+0000","last_undegraded":"2024-04-25T16:37:45.100496+0000","last_fullsized":"2024-04-25T16:37:45.100496+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:30.552997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'4","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204519+0000","last_change":"2024-04-25T16:36:26.212892+0000","last_active":"2024-04-25T16:37:16.204519+0000","last_peered":"2024-04-25T16:37:16.204519+0000","last_clean":"2024-04-25T16:37:16.204519+0000","last_became_active":"2024-04-25T16:36:26.212443+0000","last_became_peered":"2024-04-25T16:36:26.212443+0000","last_unstale":"2024-04-25T16:37:16.204519+0000","last_undegraded":"2024-04-25T16:37:16.204519+0000","last_fullsized":"2024-04-25T16:37:16.204519+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:25.168042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.367884+0000","last_change":"2024-04-25T16:37:16.367884+0000","last_active":"2024-04-25T16:37:16.367884+0000","last_peered":"2024-04-25T16:37:16.367884+0000","last_clean":"2024-04-25T16:37:16.367884+0000","last_became_active":"2024-04-25T16:37:16.367470+0000","last_became_peered":"2024-04-25T16:37:16.367470+0000","last_unstale":"2024-04-25T16:37:16.367884+0000","last_undegraded":"2024-04-25T16:37:16.367884+0000","last_fullsized":"2024-04-25T16:37:16.367884+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:46.290254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206232+0000","last_change":"2024-04-25T16:36:26.212141+0000","last_active":"2024-04-25T16:37:16.206232+0000","last_peered":"2024-04-25T16:37:16.206232+0000","last_clean":"2024-04-25T16:37:16.206232+0000","last_became_active":"2024-04-25T16:36:26.211554+0000","last_became_peered":"2024-04-25T16:36:26.211554+0000","last_unstale":"2024-04-25T16:37:16.206232+0000","last_undegraded":"2024-04-25T16:37:16.206232+0000","last_fullsized":"2024-04-25T16:37:16.206232+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:20:43.970994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.054541+0000","last_change":"2024-04-25T16:36:26.978201+0000","last_active":"2024-04-25T16:37:45.054541+0000","last_peered":"2024-04-25T16:37:45.054541+0000","last_clean":"2024-04-25T16:37:45.054541+0000","last_became_active":"2024-04-25T16:36:26.977805+0000","last_became_peered":"2024-04-25T16:36:26.977805+0000","last_unstale":"2024-04-25T16:37:45.054541+0000","last_undegraded":"2024-04-25T16:37:45.054541+0000","last_fullsized":"2024-04-25T16:37:45.054541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:23.852029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'3","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.366460+0000","last_change":"2024-04-25T16:36:26.211558+0000","last_active":"2024-04-25T16:37:16.366460+0000","last_peered":"2024-04-25T16:37:16.366460+0000","last_clean":"2024-04-25T16:37:16.366460+0000","last_became_active":"2024-04-25T16:36:26.211191+0000","last_became_peered":"2024-04-25T16:36:26.211191+0000","last_unstale":"2024-04-25T16:37:16.366460+0000","last_undegraded":"2024-04-25T16:37:16.366460+0000","last_fullsized":"2024-04-25T16:37:16.366460+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:54.031297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.099654+0000","last_change":"2024-04-25T16:36:26.987439+0000","last_active":"2024-04-25T16:37:45.099654+0000","last_peered":"2024-04-25T16:37:45.099654+0000","last_clean":"2024-04-25T16:37:45.099654+0000","last_became_active":"2024-04-25T16:36:26.986998+0000","last_became_peered":"2024-04-25T16:36:26.986998+0000","last_unstale":"2024-04-25T16:37:45.099654+0000","last_undegraded":"2024-04-25T16:37:45.099654+0000","last_fullsized":"2024-04-25T16:37:45.099654+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:49.057303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'1","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.209127+0000","last_change":"2024-04-25T16:36:26.206662+0000","last_active":"2024-04-25T16:37:16.209127+0000","last_peered":"2024-04-25T16:37:16.209127+0000","last_clean":"2024-04-25T16:37:16.209127+0000","last_became_active":"2024-04-25T16:36:26.206237+0000","last_became_peered":"2024-04-25T16:36:26.206237+0000","last_unstale":"2024-04-25T16:37:16.209127+0000","last_undegraded":"2024-04-25T16:37:16.209127+0000","last_fullsized":"2024-04-25T16:37:16.209127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:52:53.602637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.092975+0000","last_change":"2024-04-25T16:36:26.983837+0000","last_active":"2024-04-25T16:37:45.092975+0000","last_peered":"2024-04-25T16:37:45.092975+0000","last_clean":"2024-04-25T16:37:45.092975+0000","last_became_active":"2024-04-25T16:36:26.983513+0000","last_became_peered":"2024-04-25T16:36:26.983513+0000","last_unstale":"2024-04-25T16:37:45.092975+0000","last_undegraded":"2024-04-25T16:37:45.092975+0000","last_fullsized":"2024-04-25T16:37:45.092975+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:46.070866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217010+0000","last_change":"2024-04-25T16:37:16.217010+0000","last_active":"2024-04-25T16:37:16.217010+0000","last_peered":"2024-04-25T16:37:16.217010+0000","last_clean":"2024-04-25T16:37:16.217010+0000","last_became_active":"2024-04-25T16:37:16.212152+0000","last_became_peered":"2024-04-25T16:37:16.212152+0000","last_unstale":"2024-04-25T16:37:16.217010+0000","last_undegraded":"2024-04-25T16:37:16.217010+0000","last_fullsized":"2024-04-25T16:37:16.217010+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:59:07.095559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210549+0000","last_change":"2024-04-25T16:36:26.970649+0000","last_active":"2024-04-25T16:37:16.210549+0000","last_peered":"2024-04-25T16:37:16.210549+0000","last_clean":"2024-04-25T16:37:16.210549+0000","last_became_active":"2024-04-25T16:36:26.970266+0000","last_became_peered":"2024-04-25T16:36:26.970266+0000","last_unstale":"2024-04-25T16:37:16.210549+0000","last_undegraded":"2024-04-25T16:37:16.210549+0000","last_fullsized":"2024-04-25T16:37:16.210549+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:32:44.055877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'3","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.209452+0000","last_change":"2024-04-25T16:36:26.212419+0000","last_active":"2024-04-25T16:37:16.209452+0000","last_peered":"2024-04-25T16:37:16.209452+0000","last_clean":"2024-04-25T16:37:16.209452+0000","last_became_active":"2024-04-25T16:36:26.211807+0000","last_became_peered":"2024-04-25T16:36:26.211807+0000","last_unstale":"2024-04-25T16:37:16.209452+0000","last_undegraded":"2024-04-25T16:37:16.209452+0000","last_fullsized":"2024-04-25T16:37:16.209452+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:02.651503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211233+0000","last_change":"2024-04-25T16:36:26.970533+0000","last_active":"2024-04-25T16:37:16.211233+0000","last_peered":"2024-04-25T16:37:16.211233+0000","last_clean":"2024-04-25T16:37:16.211233+0000","last_became_active":"2024-04-25T16:36:26.970194+0000","last_became_peered":"2024-04-25T16:36:26.970194+0000","last_unstale":"2024-04-25T16:37:16.211233+0000","last_undegraded":"2024-04-25T16:37:16.211233+0000","last_fullsized":"2024-04-25T16:37:16.211233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:41.798930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"74'1","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.204345+0000","last_change":"2024-04-25T16:36:26.210227+0000","last_active":"2024-04-25T16:37:16.204345+0000","last_peered":"2024-04-25T16:37:16.204345+0000","last_clean":"2024-04-25T16:37:16.204345+0000","last_became_active":"2024-04-25T16:36:26.209854+0000","last_became_peered":"2024-04-25T16:36:26.209854+0000","last_unstale":"2024-04-25T16:37:16.204345+0000","last_undegraded":"2024-04-25T16:37:16.204345+0000","last_fullsized":"2024-04-25T16:37:16.204345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:48.542031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.210738+0000","last_change":"2024-04-25T16:36:26.971278+0000","last_active":"2024-04-25T16:37:16.210738+0000","last_peered":"2024-04-25T16:37:16.210738+0000","last_clean":"2024-04-25T16:37:16.210738+0000","last_became_active":"2024-04-25T16:36:26.970959+0000","last_became_peered":"2024-04-25T16:36:26.970959+0000","last_unstale":"2024-04-25T16:37:16.210738+0000","last_undegraded":"2024-04-25T16:37:16.210738+0000","last_fullsized":"2024-04-25T16:37:16.210738+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:24.886523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"74'2","reported_seq":21,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206320+0000","last_change":"2024-04-25T16:36:26.204435+0000","last_active":"2024-04-25T16:37:16.206320+0000","last_peered":"2024-04-25T16:37:16.206320+0000","last_clean":"2024-04-25T16:37:16.206320+0000","last_became_active":"2024-04-25T16:36:26.203592+0000","last_became_peered":"2024-04-25T16:36:26.203592+0000","last_unstale":"2024-04-25T16:37:16.206320+0000","last_undegraded":"2024-04-25T16:37:16.206320+0000","last_fullsized":"2024-04-25T16:37:16.206320+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:25.187846+0000","last_clean_scrub_stamp":"2024-04-25T16:36:25.187846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:18.623888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.211261+0000","last_change":"2024-04-25T16:36:26.971531+0000","last_active":"2024-04-25T16:37:16.211261+0000","last_peered":"2024-04-25T16:37:16.211261+0000","last_clean":"2024-04-25T16:37:16.211261+0000","last_became_active":"2024-04-25T16:36:26.969904+0000","last_became_peered":"2024-04-25T16:36:26.969904+0000","last_unstale":"2024-04-25T16:37:16.211261+0000","last_undegraded":"2024-04-25T16:37:16.211261+0000","last_fullsized":"2024-04-25T16:37:16.211261+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:22:47.468692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.099521+0000","last_change":"2024-04-25T16:36:26.981281+0000","last_active":"2024-04-25T16:37:45.099521+0000","last_peered":"2024-04-25T16:37:45.099521+0000","last_clean":"2024-04-25T16:37:45.099521+0000","last_became_active":"2024-04-25T16:36:26.980942+0000","last_became_peered":"2024-04-25T16:36:26.980942+0000","last_unstale":"2024-04-25T16:37:45.099521+0000","last_undegraded":"2024-04-25T16:37:45.099521+0000","last_fullsized":"2024-04-25T16:37:45.099521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:30.713519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.087853+0000","last_change":"2024-04-25T16:36:26.989589+0000","last_active":"2024-04-25T16:37:45.087853+0000","last_peered":"2024-04-25T16:37:45.087853+0000","last_clean":"2024-04-25T16:37:45.087853+0000","last_became_active":"2024-04-25T16:36:26.989127+0000","last_became_peered":"2024-04-25T16:36:26.989127+0000","last_unstale":"2024-04-25T16:37:45.087853+0000","last_undegraded":"2024-04-25T16:37:45.087853+0000","last_fullsized":"2024-04-25T16:37:45.087853+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:53.989995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.092526+0000","last_change":"2024-04-25T16:36:26.983784+0000","last_active":"2024-04-25T16:37:45.092526+0000","last_peered":"2024-04-25T16:37:45.092526+0000","last_clean":"2024-04-25T16:37:45.092526+0000","last_became_active":"2024-04-25T16:36:26.983256+0000","last_became_peered":"2024-04-25T16:36:26.983256+0000","last_unstale":"2024-04-25T16:37:45.092526+0000","last_undegraded":"2024-04-25T16:37:45.092526+0000","last_fullsized":"2024-04-25T16:37:45.092526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:44.469717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.207366+0000","last_change":"2024-04-25T16:36:26.986223+0000","last_active":"2024-04-25T16:37:16.207366+0000","last_peered":"2024-04-25T16:37:16.207366+0000","last_clean":"2024-04-25T16:37:16.207366+0000","last_became_active":"2024-04-25T16:36:26.985511+0000","last_became_peered":"2024-04-25T16:36:26.985511+0000","last_unstale":"2024-04-25T16:37:16.207366+0000","last_undegraded":"2024-04-25T16:37:16.207366+0000","last_fullsized":"2024-04-25T16:37:16.207366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:46.269298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.217428+0000","last_change":"2024-04-25T16:37:16.217428+0000","last_active":"2024-04-25T16:37:16.217428+0000","last_peered":"2024-04-25T16:37:16.217428+0000","last_clean":"2024-04-25T16:37:16.217428+0000","last_became_active":"2024-04-25T16:37:16.216975+0000","last_became_peered":"2024-04-25T16:37:16.216975+0000","last_unstale":"2024-04-25T16:37:16.217428+0000","last_undegraded":"2024-04-25T16:37:16.217428+0000","last_fullsized":"2024-04-25T16:37:16.217428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:56.498477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.094824+0000","last_change":"2024-04-25T16:36:26.974556+0000","last_active":"2024-04-25T16:37:45.094824+0000","last_peered":"2024-04-25T16:37:45.094824+0000","last_clean":"2024-04-25T16:37:45.094824+0000","last_became_active":"2024-04-25T16:36:26.974179+0000","last_became_peered":"2024-04-25T16:36:26.974179+0000","last_unstale":"2024-04-25T16:37:45.094824+0000","last_undegraded":"2024-04-25T16:37:45.094824+0000","last_fullsized":"2024-04-25T16:37:45.094824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:35.313778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.112774+0000","last_change":"2024-04-25T16:36:26.985614+0000","last_active":"2024-04-25T16:37:45.112774+0000","last_peered":"2024-04-25T16:37:45.112774+0000","last_clean":"2024-04-25T16:37:45.112774+0000","last_became_active":"2024-04-25T16:36:26.985259+0000","last_became_peered":"2024-04-25T16:36:26.985259+0000","last_unstale":"2024-04-25T16:37:45.112774+0000","last_undegraded":"2024-04-25T16:37:45.112774+0000","last_fullsized":"2024-04-25T16:37:45.112774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.850086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.083631+0000","last_change":"2024-04-25T16:36:26.981898+0000","last_active":"2024-04-25T16:37:45.083631+0000","last_peered":"2024-04-25T16:37:45.083631+0000","last_clean":"2024-04-25T16:37:45.083631+0000","last_became_active":"2024-04-25T16:36:26.981554+0000","last_became_peered":"2024-04-25T16:36:26.981554+0000","last_unstale":"2024-04-25T16:37:45.083631+0000","last_undegraded":"2024-04-25T16:37:45.083631+0000","last_fullsized":"2024-04-25T16:37:45.083631+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:20.836846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.203968+0000","last_change":"2024-04-25T16:36:26.970983+0000","last_active":"2024-04-25T16:37:16.203968+0000","last_peered":"2024-04-25T16:37:16.203968+0000","last_clean":"2024-04-25T16:37:16.203968+0000","last_became_active":"2024-04-25T16:36:26.970644+0000","last_became_peered":"2024-04-25T16:36:26.970644+0000","last_unstale":"2024-04-25T16:37:16.203968+0000","last_undegraded":"2024-04-25T16:37:16.203968+0000","last_fullsized":"2024-04-25T16:37:16.203968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:33.285236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.089374+0000","last_change":"2024-04-25T16:36:26.982347+0000","last_active":"2024-04-25T16:37:45.089374+0000","last_peered":"2024-04-25T16:37:45.089374+0000","last_clean":"2024-04-25T16:37:45.089374+0000","last_became_active":"2024-04-25T16:36:26.981970+0000","last_became_peered":"2024-04-25T16:36:26.981970+0000","last_unstale":"2024-04-25T16:37:45.089374+0000","last_undegraded":"2024-04-25T16:37:45.089374+0000","last_fullsized":"2024-04-25T16:37:45.089374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:43.589483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206332+0000","last_change":"2024-04-25T16:36:26.975329+0000","last_active":"2024-04-25T16:37:16.206332+0000","last_peered":"2024-04-25T16:37:16.206332+0000","last_clean":"2024-04-25T16:37:16.206332+0000","last_became_active":"2024-04-25T16:36:26.974791+0000","last_became_peered":"2024-04-25T16:36:26.974791+0000","last_unstale":"2024-04-25T16:37:16.206332+0000","last_undegraded":"2024-04-25T16:37:16.206332+0000","last_fullsized":"2024-04-25T16:37:16.206332+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:43.759074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:16.206186+0000","last_change":"2024-04-25T16:36:26.983841+0000","last_active":"2024-04-25T16:37:16.206186+0000","last_peered":"2024-04-25T16:37:16.206186+0000","last_clean":"2024-04-25T16:37:16.206186+0000","last_became_active":"2024-04-25T16:36:26.982595+0000","last_became_peered":"2024-04-25T16:36:26.982595+0000","last_unstale":"2024-04-25T16:37:16.206186+0000","last_undegraded":"2024-04-25T16:37:16.206186+0000","last_fullsized":"2024-04-25T16:37:16.206186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.793016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-04-25T16:37:45.098582+0000","last_change":"2024-04-25T16:36:26.984713+0000","last_active":"2024-04-25T16:37:45.098582+0000","last_peered":"2024-04-25T16:37:45.098582+0000","last_clean":"2024-04-25T16:37:45.098582+0000","last_became_active":"2024-04-25T16:36:26.984330+0000","last_became_peered":"2024-04-25T16:36:26.984330+0000","last_unstale":"2024-04-25T16:37:45.098582+0000","last_undegraded":"2024-04-25T16:37:45.098582+0000","last_fullsized":"2024-04-25T16:37:45.098582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T16:36:26.189764+0000","last_clean_scrub_stamp":"2024-04-25T16:36:26.189764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:33.028204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":246733,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65047,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":300,"num_write":156,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":195531,"data_compressed":5601,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710750,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":831488,"data_stored":401650,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906979,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1054155,"data_compressed":43355,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":843776,"data_stored":401864,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266733,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":839680,"data_stored":401762,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":993714,"data_compressed":41488,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":831488,"data_stored":401253,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":888832,"data_stored":462092,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":402524,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":463833,"data_compressed":7644,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":847872,"data_stored":401564,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986242,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":401690,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822717,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":993398,"data_compressed":41488,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35711,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63177,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62624,"data_compressed":1867,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T16:37:47.988 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T16:37:47.988 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T16:37:47.988 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T16:37:47.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T16:37:48.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:47 smithi116 ceph-mon[32712]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:37:48.227 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:48.685 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:48 smithi112 ceph-mon[29122]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:37:48.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:48 smithi133 ceph-mon[32691]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:37:49.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:48 smithi116 ceph-mon[32712]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T16:37:49.635 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:49 smithi112 ceph-mon[29122]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 -- 172.21.15.112:0/4293651764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105c50 msgr2=0x7fd7f010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 --2- 172.21.15.112:0/4293651764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105c50 0x7fd7f010e1a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009990 tx=0x7fd7e4009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:49.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 -- 172.21.15.112:0/4293651764 shutdown_connections 2024-04-25T16:37:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 --2- 172.21.15.112:0/4293651764 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f010e770 0x7fd7f0110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 --2- 172.21.15.112:0/4293651764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f0105300 0x7fd7f0105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 --2- 172.21.15.112:0/4293651764 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105c50 0x7fd7f010e1a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 -- 172.21.15.112:0/4293651764 >> 172.21.15.112:0/4293651764 conn(0x7fd7f00fecb0 msgr2=0x7fd7f01010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 -- 172.21.15.112:0/4293651764 shutdown_connections 2024-04-25T16:37:49.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 -- 172.21.15.112:0/4293651764 wait complete. 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.886+0000 7fd7f559f700 1 Processor -- start 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 -- start start 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 0x7fd7f01a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0105c50 0x7fd7f01a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f010e770 0x7fd7f019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0114350 con 0x7fd7f0105300 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01141d0 con 0x7fd7f010e770 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7f559f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01144d0 con 0x7fd7f0105c50 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 0x7fd7f01a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 0x7fd7f01a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48988/0 (socket says 172.21.15.112:48988) 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 -- 172.21.15.112:0/975020434 learned_addr learned my addr 172.21.15.112:0/975020434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7ef7fe700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f010e770 0x7fd7f019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7ee7fc700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0105c50 0x7fd7f01a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:49.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 -- 172.21.15.112:0/975020434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0105c50 msgr2=0x7fd7f01a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0105c50 0x7fd7f01a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 -- 172.21.15.112:0/975020434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f010e770 msgr2=0x7fd7f019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f010e770 0x7fd7f019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.887+0000 7fd7eeffd700 1 -- 172.21.15.112:0/975020434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f019b1d0 con 0x7fd7f0105300 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.888+0000 7fd7eeffd700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 0x7fd7f01a08b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000d950 tx=0x7fd7e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.888+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0016070 con 0x7fd7f0105300 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.888+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f019b4c0 con 0x7fd7f0105300 2024-04-25T16:37:49.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.888+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e000fd40 con 0x7fd7f0105300 2024-04-25T16:37:49.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.888+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000e3b0 con 0x7fd7f0105300 2024-04-25T16:37:49.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.888+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a7760 con 0x7fd7f0105300 2024-04-25T16:37:49.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.889+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f00761a0 con 0x7fd7f0105300 2024-04-25T16:37:49.889 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.891+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fd7e0019070 con 0x7fd7f0105300 2024-04-25T16:37:49.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.891+0000 7fd7d7fff700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd7d80afb80 0x7fd7d80b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:49.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.892+0000 7fd7ee7fc700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd7d80afb80 0x7fd7d80b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:49.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.892+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd7e00de730 con 0x7fd7f0105300 2024-04-25T16:37:49.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.892+0000 7fd7ee7fc700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd7d80afb80 0x7fd7d80b2090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400b5c0 tx=0x7fd7e4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:49.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:49.896+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd7e00a72f0 con 0x7fd7f0105300 2024-04-25T16:37:49.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:49 smithi133 ceph-mon[32691]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:50.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:49 smithi116 ceph-mon[32712]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:50.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.138+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd7f002e510 con 0x7fd7f0105300 2024-04-25T16:37:50.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.138+0000 7fd7d7fff700 1 -- 172.21.15.112:0/975020434 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd7e00af370 con 0x7fd7f0105300 2024-04-25T16:37:50.137 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:50.137 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T16:37:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd7d80afb80 msgr2=0x7fd7d80b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd7d80afb80 0x7fd7d80b2090 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd7e400b5c0 tx=0x7fd7e4005960 comp rx=0 tx=0).stop 2024-04-25T16:37:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 msgr2=0x7fd7f01a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 0x7fd7f01a08b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000d950 tx=0x7fd7e000dc60 comp rx=0 tx=0).stop 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 shutdown_connections 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd7d80afb80 0x7fd7d80b2090 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f010e770 0x7fd7f019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7f0105300 0x7fd7f01a08b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 --2- 172.21.15.112:0/975020434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7f0105c50 0x7fd7f01a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 >> 172.21.15.112:0/975020434 conn(0x7fd7f00fecb0 msgr2=0x7fd7f00ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 shutdown_connections 2024-04-25T16:37:50.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.141+0000 7fd7f559f700 1 -- 172.21.15.112:0/975020434 wait complete. 2024-04-25T16:37:50.816 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:50 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/975020434' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T16:37:50.846 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T16:37:50.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:37:50.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:50 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/975020434' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 -- 172.21.15.112:0/2553653187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 msgr2=0x7f560c104030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 --2- 172.21.15.112:0/2553653187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c104030 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5600009900 tx=0x7f560002a410 comp rx=0 tx=0).stop 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 -- 172.21.15.112:0/2553653187 shutdown_connections 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 --2- 172.21.15.112:0/2553653187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f560c108920 0x7f560c10ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 --2- 172.21.15.112:0/2553653187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c104030 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 --2- 172.21.15.112:0/2553653187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560c1029e0 0x7f560c1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 -- 172.21.15.112:0/2553653187 >> 172.21.15.112:0/2553653187 conn(0x7f560c0fdbd0 msgr2=0x7f560c100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:50.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 -- 172.21.15.112:0/2553653187 shutdown_connections 2024-04-25T16:37:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.959+0000 7f5611ae2640 1 -- 172.21.15.112:0/2553653187 wait complete. 2024-04-25T16:37:50.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 Processor -- start 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 -- start start 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f560c1029e0 0x7f560c075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560c108920 0x7f560c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10e4e0 con 0x7f560c103c20 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10e360 con 0x7f560c1029e0 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f5611ae2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10e660 con 0x7f560c108920 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f560affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f560affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:48996/0 (socket says 172.21.15.112:48996) 2024-04-25T16:37:50.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f560affd640 1 -- 172.21.15.112:0/3492447917 learned_addr learned my addr 172.21.15.112:0/3492447917 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.960+0000 7f560b7fe640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f560c1029e0 0x7f560c075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560bfff640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560c108920 0x7f560c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560affd640 1 -- 172.21.15.112:0/3492447917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560c108920 msgr2=0x7f560c1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560affd640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560c108920 0x7f560c1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560affd640 1 -- 172.21.15.112:0/3492447917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f560c1029e0 msgr2=0x7f560c075840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560affd640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f560c1029e0 0x7f560c075840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560affd640 1 -- 172.21.15.112:0/3492447917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1a1d10 con 0x7f560c103c20 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f560affd640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c19efa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5600006010 tx=0x7f5600004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560003f030 con 0x7f560c103c20 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5600003cb0 con 0x7f560c103c20 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1a1fa0 con 0x7f560c103c20 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600035070 con 0x7f560c103c20 2024-04-25T16:37:50.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.961+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1a23b0 con 0x7f560c103c20 2024-04-25T16:37:50.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.962+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c107730 con 0x7f560c103c20 2024-04-25T16:37:50.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.963+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f5600042030 con 0x7f560c103c20 2024-04-25T16:37:50.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.964+0000 7f5608ff9640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f55f40afaf0 0x7f55f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:50.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.965+0000 7f560b7fe640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f55f40afaf0 0x7f55f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:50.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.965+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f56000ff9a0 con 0x7f560c103c20 2024-04-25T16:37:50.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.965+0000 7f560b7fe640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f55f40afaf0 0x7f55f40b2000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f560c1035e0 tx=0x7f55fc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:50.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:50.968+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56000c8500 con 0x7f560c103c20 2024-04-25T16:37:51.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:50 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/975020434' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T16:37:51.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.154+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f560c02dd20 con 0x7f560c103c20 2024-04-25T16:37:51.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.156+0000 7f5608ff9640 1 -- 172.21.15.112:0/3492447917 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22824 (secure 0 0 0) 0x7f5600003e60 con 0x7f560c103c20 2024-04-25T16:37:51.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:51.155 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":77,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:37:16.199786+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f55f40afaf0 msgr2=0x7f55f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f55f40afaf0 0x7f55f40b2000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f560c1035e0 tx=0x7f55fc00b390 comp rx=0 tx=0).stop 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 msgr2=0x7f560c19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c19efa0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5600006010 tx=0x7f5600004340 comp rx=0 tx=0).stop 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 shutdown_connections 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f55f40afaf0 0x7f55f40b2000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560c108920 0x7f560c1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f560c103c20 0x7f560c19efa0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 --2- 172.21.15.112:0/3492447917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f560c1029e0 0x7f560c075840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 >> 172.21.15.112:0/3492447917 conn(0x7f560c0fdbd0 msgr2=0x7f560c104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:51.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 shutdown_connections 2024-04-25T16:37:51.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.158+0000 7f5611ae2640 1 -- 172.21.15.112:0/3492447917 wait complete. 2024-04-25T16:37:51.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:36:24.204313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:36:25.525471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:37:51.169 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:37:51.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 -- 172.21.15.112:0/1904585835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52441050a0 msgr2=0x7f52441054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 --2- 172.21.15.112:0/1904585835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52441050a0 0x7f52441054b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5238009900 tx=0x7f523802a410 comp rx=0 tx=0).stop 2024-04-25T16:37:51.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 -- 172.21.15.112:0/1904585835 shutdown_connections 2024-04-25T16:37:51.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 --2- 172.21.15.112:0/1904585835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5244101750 0x7f5244101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 --2- 172.21.15.112:0/1904585835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5244102250 0x7f524410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 --2- 172.21.15.112:0/1904585835 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52441050a0 0x7f52441054b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 -- 172.21.15.112:0/1904585835 >> 172.21.15.112:0/1904585835 conn(0x7f52440fd350 msgr2=0x7f52440ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 -- 172.21.15.112:0/1904585835 shutdown_connections 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 -- 172.21.15.112:0/1904585835 wait complete. 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.284+0000 7f52495c0640 1 Processor -- start 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 -- start start 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5244101750 0x7f524419a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 0x7f524419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52441050a0 0x7f52441a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f524410f5f0 con 0x7f5244102250 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f524410f470 con 0x7f52441050a0 2024-04-25T16:37:51.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52495c0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f524410f770 con 0x7f5244101750 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 0x7f524419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 0x7f524419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49006/0 (socket says 172.21.15.112:49006) 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 -- 172.21.15.112:0/4133469842 learned_addr learned my addr 172.21.15.112:0/4133469842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f5242ffd640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5244101750 0x7f524419a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 -- 172.21.15.112:0/4133469842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5244101750 msgr2=0x7f524419a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5244101750 0x7f524419a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 -- 172.21.15.112:0/4133469842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52441050a0 msgr2=0x7f52441a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52441050a0 0x7f52441a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 -- 172.21.15.112:0/4133469842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52441a1d60 con 0x7f5244102250 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.285+0000 7f52427fc640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 0x7f524419a760 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f523400d950 tx=0x7f523400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.286+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234017070 con 0x7f5244102250 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.286+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5234009c30 con 0x7f5244102250 2024-04-25T16:37:51.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.286+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441a1ff0 con 0x7f5244102250 2024-04-25T16:37:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.286+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523400f620 con 0x7f5244102250 2024-04-25T16:37:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.286+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a24e0 con 0x7f5244102250 2024-04-25T16:37:51.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.287+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5244108a30 con 0x7f5244102250 2024-04-25T16:37:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.288+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f5234010460 con 0x7f5244102250 2024-04-25T16:37:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.289+0000 7f522bfff640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f522c0afaf0 0x7f522c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.289+0000 7f5242ffd640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f522c0afaf0 0x7f522c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.289+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f52340de920 con 0x7f5244102250 2024-04-25T16:37:51.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.289+0000 7f5242ffd640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f522c0afaf0 0x7f522c0b2000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f523802a930 tx=0x7f5238005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:51.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.292+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52340a74e0 con 0x7f5244102250 2024-04-25T16:37:51.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.474+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f524402dda0 con 0x7f5244102250 2024-04-25T16:37:51.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.474+0000 7f522bfff640 1 -- 172.21.15.112:0/4133469842 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f5234009de0 con 0x7f5244102250 2024-04-25T16:37:51.473 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f522c0afaf0 msgr2=0x7f522c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f522c0afaf0 0x7f522c0b2000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f523802a930 tx=0x7f5238005fd0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 msgr2=0x7f524419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 0x7f524419a760 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f523400d950 tx=0x7f523400de10 comp rx=0 tx=0).stop 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 shutdown_connections 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52441050a0 0x7f52441a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5244102250 0x7f524419a760 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f522c0afaf0 0x7f522c0b2000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 --2- 172.21.15.112:0/4133469842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5244101750 0x7f524419a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.476+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 >> 172.21.15.112:0/4133469842 conn(0x7f52440fd350 msgr2=0x7f5244105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:51.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.477+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 shutdown_connections 2024-04-25T16:37:51.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.477+0000 7f52495c0640 1 -- 172.21.15.112:0/4133469842 wait complete. 2024-04-25T16:37:51.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:37:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 -- 172.21.15.112:0/1612958942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac301045f0 msgr2=0x7fac30106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 --2- 172.21.15.112:0/1612958942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac301045f0 0x7fac30106a10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fac20009900 tx=0x7fac2002a410 comp rx=0 tx=0).stop 2024-04-25T16:37:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 -- 172.21.15.112:0/1612958942 shutdown_connections 2024-04-25T16:37:51.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 --2- 172.21.15.112:0/1612958942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac30106f50 0x7fac30109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 --2- 172.21.15.112:0/1612958942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac301098b0 0x7fac3010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 --2- 172.21.15.112:0/1612958942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac301045f0 0x7fac30106a10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 -- 172.21.15.112:0/1612958942 >> 172.21.15.112:0/1612958942 conn(0x7fac300fdbf0 msgr2=0x7fac301000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 -- 172.21.15.112:0/1612958942 shutdown_connections 2024-04-25T16:37:51.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.599+0000 7fac34c70640 1 -- 172.21.15.112:0/1612958942 wait complete. 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.600+0000 7fac34c70640 1 Processor -- start 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.600+0000 7fac34c70640 1 -- start start 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac301045f0 0x7fac3019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 0x7fac3019a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac301098b0 0x7fac301a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fac3010f6a0 con 0x7fac30106f50 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fac3010f520 con 0x7fac301045f0 2024-04-25T16:37:51.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fac3010f820 con 0x7fac301098b0 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 0x7fac3019a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 0x7fac3019a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49024/0 (socket says 172.21.15.112:49024) 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 -- 172.21.15.112:0/1076354660 learned_addr learned my addr 172.21.15.112:0/1076354660 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac2e59c640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac301045f0 0x7fac3019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac2ed9d640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac301098b0 0x7fac301a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 -- 172.21.15.112:0/1076354660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac301098b0 msgr2=0x7fac301a17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac301098b0 0x7fac301a17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 -- 172.21.15.112:0/1076354660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac301045f0 msgr2=0x7fac3019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac301045f0 0x7fac3019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 -- 172.21.15.112:0/1076354660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac301a1e10 con 0x7fac30106f50 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac27fff640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 0x7fac3019a810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fac1800d770 tx=0x7fac1800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18017070 con 0x7fac30106f50 2024-04-25T16:37:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac180054d0 con 0x7fac30106f50 2024-04-25T16:37:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac301a2100 con 0x7fac30106f50 2024-04-25T16:37:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac18004c30 con 0x7fac30106f50 2024-04-25T16:37:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.601+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301a26d0 con 0x7fac30106f50 2024-04-25T16:37:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.603+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac300ff710 con 0x7fac30106f50 2024-04-25T16:37:51.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.604+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fac18004e10 con 0x7fac30106f50 2024-04-25T16:37:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.604+0000 7fac277fe640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fac1c0afb70 0x7fac1c0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.604+0000 7fac2e59c640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fac1c0afb70 0x7fac1c0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.604+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fac180de6c0 con 0x7fac30106f50 2024-04-25T16:37:51.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.605+0000 7fac2e59c640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fac1c0afb70 0x7fac1c0b2080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fac20009230 tx=0x7fac20031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:51.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.609+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac180a7280 con 0x7fac30106f50 2024-04-25T16:37:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.788+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac300637a0 con 0x7fac30106f50 2024-04-25T16:37:51.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.788+0000 7fac277fe640 1 -- 172.21.15.112:0/1076354660 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fac180a6b80 con 0x7fac30106f50 2024-04-25T16:37:51.787 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T16:37:51.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fac1c0afb70 msgr2=0x7fac1c0b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fac1c0afb70 0x7fac1c0b2080 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fac20009230 tx=0x7fac20031400 comp rx=0 tx=0).stop 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 msgr2=0x7fac3019a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 0x7fac3019a810 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fac1800d770 tx=0x7fac1800dc30 comp rx=0 tx=0).stop 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 shutdown_connections 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fac1c0afb70 0x7fac1c0b2080 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac301098b0 0x7fac301a17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac30106f50 0x7fac3019a810 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 --2- 172.21.15.112:0/1076354660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac301045f0 0x7fac3019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 >> 172.21.15.112:0/1076354660 conn(0x7fac300fdbf0 msgr2=0x7fac301051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 shutdown_connections 2024-04-25T16:37:51.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.790+0000 7fac34c70640 1 -- 172.21.15.112:0/1076354660 wait complete. 2024-04-25T16:37:51.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:51 smithi112 ceph-mon[29122]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T16:37:51.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:51 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3492447917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:51.799 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:51 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4133469842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:37:51.800 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:37:51.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 -- 172.21.15.112:0/1863421097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd38101760 msgr2=0x7efd38101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 --2- 172.21.15.112:0/1863421097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd38101760 0x7efd38101bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efd20009900 tx=0x7efd2002a410 comp rx=0 tx=0).stop 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 -- 172.21.15.112:0/1863421097 shutdown_connections 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 --2- 172.21.15.112:0/1863421097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd38102260 0x7efd3810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 --2- 172.21.15.112:0/1863421097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd38101760 0x7efd38101bd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 --2- 172.21.15.112:0/1863421097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd381050b0 0x7efd381054c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 -- 172.21.15.112:0/1863421097 >> 172.21.15.112:0/1863421097 conn(0x7efd380fd380 msgr2=0x7efd380ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 -- 172.21.15.112:0/1863421097 shutdown_connections 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.912+0000 7efd3dd84640 1 -- 172.21.15.112:0/1863421097 wait complete. 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 Processor -- start 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 -- start start 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd38101760 0x7efd38104260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd38102260 0x7efd38102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 0x7efd38102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd3810f620 con 0x7efd381050b0 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd3810f4a0 con 0x7efd38101760 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd3dd84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd3810f7a0 con 0x7efd38102260 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 0x7efd38102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 0x7efd38102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49032/0 (socket says 172.21.15.112:49032) 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 -- 172.21.15.112:0/3876707824 learned_addr learned my addr 172.21.15.112:0/3876707824 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd377fe640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd38101760 0x7efd38104260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd2ffff640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd38102260 0x7efd38102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 -- 172.21.15.112:0/3876707824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd38102260 msgr2=0x7efd38102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd38102260 0x7efd38102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 -- 172.21.15.112:0/3876707824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd38101760 msgr2=0x7efd38104260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:51.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd38101760 0x7efd38104260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 -- 172.21.15.112:0/3876707824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd38103570 con 0x7efd381050b0 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.913+0000 7efd37fff640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 0x7efd38102e40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd30009fd0 tx=0x7efd3000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.914+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30017070 con 0x7efd381050b0 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.914+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd30003e20 con 0x7efd381050b0 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.914+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd381a4340 con 0x7efd381050b0 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.914+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30010460 con 0x7efd381050b0 2024-04-25T16:37:51.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.914+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd381a47b0 con 0x7efd381050b0 2024-04-25T16:37:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.915+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd38108a40 con 0x7efd381050b0 2024-04-25T16:37:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.916+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7efd30003970 con 0x7efd381050b0 2024-04-25T16:37:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.917+0000 7efd357fa640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efd240afaf0 0x7efd240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.917+0000 7efd377fe640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efd240afaf0 0x7efd240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.917+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7efd300dea90 con 0x7efd381050b0 2024-04-25T16:37:51.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.917+0000 7efd377fe640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efd240afaf0 0x7efd240b2000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efd3806b2d0 tx=0x7efd2800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:51.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:51.921+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd300a7650 con 0x7efd381050b0 2024-04-25T16:37:51.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:51 smithi133 ceph-mon[32691]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T16:37:51.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:51 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3492447917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:51.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:51 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4133469842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:37:52.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:51 smithi116 ceph-mon[32712]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T16:37:52.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:51 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3492447917' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:52.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:51 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4133469842' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:37:52.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.099+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efd3802db50 con 0x7efd381050b0 2024-04-25T16:37:52.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.099+0000 7efd357fa640 1 -- 172.21.15.112:0/3876707824 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7efd300a6f50 con 0x7efd381050b0 2024-04-25T16:37:52.098 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efd240afaf0 msgr2=0x7efd240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efd240afaf0 0x7efd240b2000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efd3806b2d0 tx=0x7efd2800a2d0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 msgr2=0x7efd38102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 0x7efd38102e40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7efd30009fd0 tx=0x7efd3000b260 comp rx=0 tx=0).stop 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 shutdown_connections 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7efd240afaf0 0x7efd240b2000 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd38102260 0x7efd38102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd381050b0 0x7efd38102e40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 --2- 172.21.15.112:0/3876707824 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd38101760 0x7efd38104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 >> 172.21.15.112:0/3876707824 conn(0x7efd380fd380 msgr2=0x7efd38105c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 shutdown_connections 2024-04-25T16:37:52.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.101+0000 7efd3dd84640 1 -- 172.21.15.112:0/3876707824 wait complete. 2024-04-25T16:37:52.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 -- 172.21.15.112:0/4271561798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 msgr2=0x7fa1d010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/4271561798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d010ad40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009ca0 tx=0x7fa1c802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 -- 172.21.15.112:0/4271561798 shutdown_connections 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/4271561798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1d0103c20 0x7fa1d0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/4271561798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d010ad40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/4271561798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d01029e0 0x7fa1d01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 -- 172.21.15.112:0/4271561798 >> 172.21.15.112:0/4271561798 conn(0x7fa1d00fdbd0 msgr2=0x7fa1d0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 -- 172.21.15.112:0/4271561798 shutdown_connections 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 -- 172.21.15.112:0/4271561798 wait complete. 2024-04-25T16:37:52.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.220+0000 7fa1d55ee640 1 Processor -- start 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 -- start start 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1d01029e0 0x7fa1d0075860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d0103c20 0x7fa1d019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d01a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010e510 con 0x7fa1d0108920 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010e390 con 0x7fa1d0103c20 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1d55ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010e690 con 0x7fa1d01029e0 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d01a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d01a14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49048/0 (socket says 172.21.15.112:49048) 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 -- 172.21.15.112:0/2918891992 learned_addr learned my addr 172.21.15.112:0/2918891992 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1ceffd640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1d01029e0 0x7fa1d0075860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 -- 172.21.15.112:0/2918891992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1d01029e0 msgr2=0x7fa1d0075860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1d01029e0 0x7fa1d0075860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1ce7fc640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d0103c20 0x7fa1d019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 -- 172.21.15.112:0/2918891992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d0103c20 msgr2=0x7fa1d019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d0103c20 0x7fa1d019ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 -- 172.21.15.112:0/2918891992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01a1cd0 con 0x7fa1d0108920 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.221+0000 7fa1cf7fe640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d01a14b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8004af0 tx=0x7fa1c8031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.222+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8003b70 con 0x7fa1d0108920 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.222+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01a1f60 con 0x7fa1d0108920 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.222+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8003d10 con 0x7fa1d0108920 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.222+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c802fb20 con 0x7fa1d0108920 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.222+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a2420 con 0x7fa1d0108920 2024-04-25T16:37:52.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.223+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d0107730 con 0x7fa1d0108920 2024-04-25T16:37:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.224+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fa1c8004580 con 0x7fa1d0108920 2024-04-25T16:37:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.225+0000 7fa1c7fff640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1bc0afaf0 0x7fa1bc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.225+0000 7fa1ceffd640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1bc0afaf0 0x7fa1bc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.225+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fa1c81005b0 con 0x7fa1d0108920 2024-04-25T16:37:52.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.225+0000 7fa1ceffd640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1bc0afaf0 0x7fa1bc0b2000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1d01035e0 tx=0x7fa1c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:52.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.229+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1c80c9170 con 0x7fa1d0108920 2024-04-25T16:37:52.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.445+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d002db50 con 0x7fa1d0108920 2024-04-25T16:37:52.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.446+0000 7fa1c7fff640 1 -- 172.21.15.112:0/2918891992 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11878 (secure 0 0 0) 0x7fa1c80c8a70 con 0x7fa1d0108920 2024-04-25T16:37:52.444 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:52.445 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:36:31.235238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24514,24524,24560]},"id":1}]} 2024-04-25T16:37:52.445 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1bc0afaf0 msgr2=0x7fa1bc0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1bc0afaf0 0x7fa1bc0b2000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa1d01035e0 tx=0x7fa1c0005f90 comp rx=0 tx=0).stop 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 msgr2=0x7fa1d01a14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d01a14b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8004af0 tx=0x7fa1c8031700 comp rx=0 tx=0).stop 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 shutdown_connections 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1d0108920 0x7fa1d01a14b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d0103c20 0x7fa1d019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1bc0afaf0 0x7fa1bc0b2000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 --2- 172.21.15.112:0/2918891992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1d01029e0 0x7fa1d0075860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 >> 172.21.15.112:0/2918891992 conn(0x7fa1d00fdbd0 msgr2=0x7fa1d0104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 shutdown_connections 2024-04-25T16:37:52.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.448+0000 7fa1d55ee640 1 -- 172.21.15.112:0/2918891992 wait complete. 2024-04-25T16:37:52.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:37:52.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 -- 172.21.15.112:0/635155081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108940 msgr2=0x7f73d410ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 --2- 172.21.15.112:0/635155081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108940 0x7f73d410ad60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009ca0 tx=0x7f73cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 -- 172.21.15.112:0/635155081 shutdown_connections 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 --2- 172.21.15.112:0/635155081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4103c40 0x7f73d4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 --2- 172.21.15.112:0/635155081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108940 0x7f73d410ad60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 --2- 172.21.15.112:0/635155081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 0x7f73d4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 -- 172.21.15.112:0/635155081 >> 172.21.15.112:0/635155081 conn(0x7f73d40fdbf0 msgr2=0x7f73d41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 -- 172.21.15.112:0/635155081 shutdown_connections 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 -- 172.21.15.112:0/635155081 wait complete. 2024-04-25T16:37:52.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 Processor -- start 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.568+0000 7f73d9c51640 1 -- start start 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 0x7f73d41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 0x7f73d41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108940 0x7f73d41a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410e530 con 0x7f73d4103c40 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410e3b0 con 0x7f73d4102a00 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410e6b0 con 0x7f73d4108940 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 0x7f73d41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 0x7f73d41995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49064/0 (socket says 172.21.15.112:49064) 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 -- 172.21.15.112:0/143375048 learned_addr learned my addr 172.21.15.112:0/143375048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:52.571 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d37fe640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 0x7f73d41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d3fff640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108940 0x7f73d41a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 -- 172.21.15.112:0/143375048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108940 msgr2=0x7f73d41a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108940 0x7f73d41a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 -- 172.21.15.112:0/143375048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 msgr2=0x7f73d41990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 0x7f73d41990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 -- 172.21.15.112:0/143375048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d41a0b30 con 0x7f73d4103c40 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d37fe640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 0x7f73d41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d3fff640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108940 0x7f73d41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73cbfff640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 0x7f73d41995e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00d770 tx=0x7f73bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc017070 con 0x7f73d4103c40 2024-04-25T16:37:52.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73bc0054d0 con 0x7f73d4103c40 2024-04-25T16:37:52.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc004c30 con 0x7f73d4103c40 2024-04-25T16:37:52.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d41a0e20 con 0x7f73d4103c40 2024-04-25T16:37:52.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.569+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a1290 con 0x7f73d4103c40 2024-04-25T16:37:52.573 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.570+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d4107750 con 0x7f73d4103c40 2024-04-25T16:37:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.572+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f73bc004dd0 con 0x7f73d4103c40 2024-04-25T16:37:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.572+0000 7f73d17fa640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afb70 0x7f73c00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.572+0000 7f73d37fe640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afb70 0x7f73c00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.572+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f73bc0deb80 con 0x7f73d4103c40 2024-04-25T16:37:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.573+0000 7f73d37fe640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afb70 0x7f73c00b2080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73d4103600 tx=0x7f73c400a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:52.574 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.576+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73bc0a77c0 con 0x7f73d4103c40 2024-04-25T16:37:52.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.754+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73d402dda0 con 0x7f73d4103c40 2024-04-25T16:37:52.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.756+0000 7f73d17fa640 1 -- 172.21.15.112:0/143375048 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22824 (secure 0 0 0) 0x7f73bc0a70c0 con 0x7f73d4103c40 2024-04-25T16:37:52.754 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:52.755 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":77,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:37:16.199786+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:37:52.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afb70 msgr2=0x7f73c00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afb70 0x7f73c00b2080 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f73d4103600 tx=0x7f73c400a5e0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 msgr2=0x7f73d41995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 0x7f73d41995e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00d770 tx=0x7f73bc00dc30 comp rx=0 tx=0).stop 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 shutdown_connections 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afb70 0x7f73c00b2080 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108940 0x7f73d41a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c40 0x7f73d41995e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 --2- 172.21.15.112:0/143375048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d4102a00 0x7f73d41990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 >> 172.21.15.112:0/143375048 conn(0x7f73d40fdbf0 msgr2=0x7f73d4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 shutdown_connections 2024-04-25T16:37:52.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.758+0000 7f73d9c51640 1 -- 172.21.15.112:0/143375048 wait complete. 2024-04-25T16:37:52.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:52 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1076354660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:37:52.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:52 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3876707824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:37:52.767 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:52 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2918891992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:52.768 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:36:24.204313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:36:25.525471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:37:52.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:37:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 -- 172.21.15.112:0/3047138628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc108920 msgr2=0x7f48cc10ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 --2- 172.21.15.112:0/3047138628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc108920 0x7f48cc10ad40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009900 tx=0x7f48c402a410 comp rx=0 tx=0).stop 2024-04-25T16:37:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 -- 172.21.15.112:0/3047138628 shutdown_connections 2024-04-25T16:37:52.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 --2- 172.21.15.112:0/3047138628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48cc103c20 0x7f48cc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 --2- 172.21.15.112:0/3047138628 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc108920 0x7f48cc10ad40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 --2- 172.21.15.112:0/3047138628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48cc1029e0 0x7f48cc1036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 -- 172.21.15.112:0/3047138628 >> 172.21.15.112:0/3047138628 conn(0x7f48cc0fdbd0 msgr2=0x7f48cc100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 -- 172.21.15.112:0/3047138628 shutdown_connections 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 -- 172.21.15.112:0/3047138628 wait complete. 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.880+0000 7f48d1292640 1 Processor -- start 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 -- start start 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 0x7f48cc1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48cc103c20 0x7f48cc1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48cc108920 0x7f48cc1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e510 con 0x7f48cc1029e0 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e390 con 0x7f48cc103c20 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48d1292640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc10e690 con 0x7f48cc108920 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 0x7f48cc1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 0x7f48cc1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49078/0 (socket says 172.21.15.112:49078) 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 -- 172.21.15.112:0/1187742064 learned_addr learned my addr 172.21.15.112:0/1187742064 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 -- 172.21.15.112:0/1187742064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48cc108920 msgr2=0x7f48cc1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48cc108920 0x7f48cc1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 -- 172.21.15.112:0/1187742064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48cc103c20 msgr2=0x7f48cc1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48cc103c20 0x7f48cc1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 -- 172.21.15.112:0/1187742064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc1a0bf0 con 0x7f48cc1029e0 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48caffd640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 0x7f48cc1990b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f48bc007c50 tx=0x7f48bc00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.881+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc019020 con 0x7f48cc1029e0 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.882+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48bc003990 con 0x7f48cc1029e0 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.882+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc06b230 con 0x7f48cc1029e0 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.882+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc004400 con 0x7f48cc1029e0 2024-04-25T16:37:52.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.882+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc06b780 con 0x7f48cc1029e0 2024-04-25T16:37:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.883+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc0692e0 con 0x7f48cc1029e0 2024-04-25T16:37:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.884+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f48bc003b40 con 0x7f48cc1029e0 2024-04-25T16:37:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.885+0000 7f48b3fff640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48b40afaf0 0x7f48b40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.885+0000 7f48ca7fc640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48b40afaf0 0x7f48b40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:52.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.885+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f48bc0e0930 con 0x7f48cc1029e0 2024-04-25T16:37:52.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.885+0000 7f48ca7fc640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48b40afaf0 0x7f48b40b2000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48cc19a150 tx=0x7f48c0005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:52.887 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:52.889+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48bc023020 con 0x7f48cc1029e0 2024-04-25T16:37:52.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:52 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1076354660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:37:52.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:52 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3876707824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:37:52.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:52 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2918891992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:53.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:52 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1076354660' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:37:53.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:52 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3876707824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:37:53.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:52 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2918891992' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.067+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48cc107410 con 0x7f48cc1029e0 2024-04-25T16:37:53.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.067+0000 7f48b3fff640 1 -- 172.21.15.112:0/1187742064 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f48bc014e50 con 0x7f48cc1029e0 2024-04-25T16:37:53.066 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48b40afaf0 msgr2=0x7f48b40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48b40afaf0 0x7f48b40b2000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f48cc19a150 tx=0x7f48c0005bf0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 msgr2=0x7f48cc1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 0x7f48cc1990b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f48bc007c50 tx=0x7f48bc00e420 comp rx=0 tx=0).stop 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 shutdown_connections 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f48b40afaf0 0x7f48b40b2000 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48cc108920 0x7f48cc1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48cc103c20 0x7f48cc1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 --2- 172.21.15.112:0/1187742064 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f48cc1029e0 0x7f48cc1990b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 >> 172.21.15.112:0/1187742064 conn(0x7f48cc0fdbd0 msgr2=0x7f48cc104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:53.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 shutdown_connections 2024-04-25T16:37:53.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.069+0000 7f48d1292640 1 -- 172.21.15.112:0/1187742064 wait complete. 2024-04-25T16:37:53.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:37:53.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 -- 172.21.15.112:0/1969896884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c30 msgr2=0x7f73d4104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/1969896884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c30 0x7f73d4104040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009900 tx=0x7f73c402a410 comp rx=0 tx=0).stop 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 -- 172.21.15.112:0/1969896884 shutdown_connections 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/1969896884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4108930 0x7f73d410ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/1969896884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4103c30 0x7f73d4104040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/1969896884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d41029f0 0x7f73d41036f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 -- 172.21.15.112:0/1969896884 >> 172.21.15.112:0/1969896884 conn(0x7f73d40fdbc0 msgr2=0x7f73d4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 -- 172.21.15.112:0/1969896884 shutdown_connections 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.190+0000 7f73dbbb2640 1 -- 172.21.15.112:0/1969896884 wait complete. 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 Processor -- start 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 -- start start 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d41029f0 0x7f73d40757f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4103c30 0x7f73d419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 0x7f73d41a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410e520 con 0x7f73d4108930 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410e3a0 con 0x7f73d41029f0 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73dbbb2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f73d410e6a0 con 0x7f73d4103c30 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 0x7f73d41a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 0x7f73d41a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49094/0 (socket says 172.21.15.112:49094) 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 -- 172.21.15.112:0/985686474 learned_addr learned my addr 172.21.15.112:0/985686474 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 -- 172.21.15.112:0/985686474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4103c30 msgr2=0x7f73d419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4103c30 0x7f73d419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 -- 172.21.15.112:0/985686474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d41029f0 msgr2=0x7f73d40757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d41029f0 0x7f73d40757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 -- 172.21.15.112:0/985686474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d41a1d30 con 0x7f73d4108930 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.191+0000 7f73da14f640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 0x7f73d41a1560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f73d000cad0 tx=0x7f73d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.192+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d000ae20 con 0x7f73d4108930 2024-04-25T16:37:53.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.192+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d41a1fc0 con 0x7f73d4108930 2024-04-25T16:37:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.192+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73d0005020 con 0x7f73d4108930 2024-04-25T16:37:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.192+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d000f6b0 con 0x7f73d4108930 2024-04-25T16:37:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.192+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d41a24e0 con 0x7f73d4108930 2024-04-25T16:37:53.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.193+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d4107740 con 0x7f73d4108930 2024-04-25T16:37:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.194+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f73d00051d0 con 0x7f73d4108930 2024-04-25T16:37:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.194+0000 7f73caffd640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afaf0 0x7f73c00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.195+0000 7f73d994e640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afaf0 0x7f73c00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.195+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f73d00df110 con 0x7f73d4108930 2024-04-25T16:37:53.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.195+0000 7f73d994e640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afaf0 0x7f73c00b2000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73d41035f0 tx=0x7f73cc005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:53.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.198+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73d0014030 con 0x7f73d4108930 2024-04-25T16:37:53.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.375+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f73d402de00 con 0x7f73d4108930 2024-04-25T16:37:53.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.375+0000 7f73caffd640 1 -- 172.21.15.112:0/985686474 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f73d00a7cd0 con 0x7f73d4108930 2024-04-25T16:37:53.375 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T16:37:53.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afaf0 msgr2=0x7f73c00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afaf0 0x7f73c00b2000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f73d41035f0 tx=0x7f73cc005c10 comp rx=0 tx=0).stop 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 msgr2=0x7f73d41a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 0x7f73d41a1560 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f73d000cad0 tx=0x7f73d000cf90 comp rx=0 tx=0).stop 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 shutdown_connections 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f73c00afaf0 0x7f73c00b2000 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73d4103c30 0x7f73d419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f73d4108930 0x7f73d41a1560 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 --2- 172.21.15.112:0/985686474 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73d41029f0 0x7f73d40757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 >> 172.21.15.112:0/985686474 conn(0x7f73d40fdbc0 msgr2=0x7f73d4104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 shutdown_connections 2024-04-25T16:37:53.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.377+0000 7f73dbbb2640 1 -- 172.21.15.112:0/985686474 wait complete. 2024-04-25T16:37:53.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:37:53.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 -- 172.21.15.112:0/2615487998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 msgr2=0x7fc00c103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 --2- 172.21.15.112:0/2615487998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c103de0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc000009900 tx=0x7fc00002a410 comp rx=0 tx=0).stop 2024-04-25T16:37:53.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 -- 172.21.15.112:0/2615487998 shutdown_connections 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 --2- 172.21.15.112:0/2615487998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00c104470 0x7fc00c10abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 --2- 172.21.15.112:0/2615487998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c103de0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 --2- 172.21.15.112:0/2615487998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c102f50 0x7fc00c103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 -- 172.21.15.112:0/2615487998 >> 172.21.15.112:0/2615487998 conn(0x7fc00c0fd350 msgr2=0x7fc00c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 -- 172.21.15.112:0/2615487998 shutdown_connections 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.499+0000 7fc012820640 1 -- 172.21.15.112:0/2615487998 wait complete. 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 Processor -- start 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 -- start start 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00c102f50 0x7fc00c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c104470 0x7fc00c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10e5a0 con 0x7fc00c103930 2024-04-25T16:37:53.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10e420 con 0x7fc00c104470 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc012820640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10e720 con 0x7fc00c102f50 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49116/0 (socket says 172.21.15.112:49116) 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 -- 172.21.15.112:0/2344950333 learned_addr learned my addr 172.21.15.112:0/2344950333 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00bfff640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00c102f50 0x7fc00c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 -- 172.21.15.112:0/2344950333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00c102f50 msgr2=0x7fc00c199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00c102f50 0x7fc00c199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 -- 172.21.15.112:0/2344950333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c104470 msgr2=0x7fc00c1a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc010dbd640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c104470 0x7fc00c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c104470 0x7fc00c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.500+0000 7fc00b7fe640 1 -- 172.21.15.112:0/2344950333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1a0c80 con 0x7fc00c103930 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.501+0000 7fc00b7fe640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c199680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc000006010 tx=0x7fc0000043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:53.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.501+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00003f030 con 0x7fc00c103930 2024-04-25T16:37:53.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.501+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc000003cb0 con 0x7fc00c103930 2024-04-25T16:37:53.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.501+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a0f10 con 0x7fc00c103930 2024-04-25T16:37:53.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.501+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000035070 con 0x7fc00c103930 2024-04-25T16:37:53.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.501+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a1390 con 0x7fc00c103930 2024-04-25T16:37:53.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.502+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c1078f0 con 0x7fc00c103930 2024-04-25T16:37:53.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.503+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fc000042030 con 0x7fc00c103930 2024-04-25T16:37:53.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.504+0000 7fc0097fa640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbff40afaf0 0x7fbff40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.504+0000 7fc00bfff640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbff40afaf0 0x7fbff40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.504+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fc0000ffad0 con 0x7fc00c103930 2024-04-25T16:37:53.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.504+0000 7fc00bfff640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbff40afaf0 0x7fbff40b2000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0058c0 tx=0x7fbffc00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:53.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.507+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0000c8690 con 0x7fc00c103930 2024-04-25T16:37:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.690+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc00c0637d0 con 0x7fc00c103930 2024-04-25T16:37:53.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.691+0000 7fc0097fa640 1 -- 172.21.15.112:0/2344950333 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7fc0000c7f90 con 0x7fc00c103930 2024-04-25T16:37:53.690 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T16:37:53.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbff40afaf0 msgr2=0x7fbff40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbff40afaf0 0x7fbff40b2000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbffc0058c0 tx=0x7fbffc00a4c0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 msgr2=0x7fc00c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c199680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc000006010 tx=0x7fc0000043b0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 shutdown_connections 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc00c104470 0x7fc00c1a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc00c103930 0x7fc00c199680 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbff40afaf0 0x7fbff40b2000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 --2- 172.21.15.112:0/2344950333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc00c102f50 0x7fc00c199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 >> 172.21.15.112:0/2344950333 conn(0x7fc00c0fd350 msgr2=0x7fc00c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 shutdown_connections 2024-04-25T16:37:53.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.694+0000 7fc012820640 1 -- 172.21.15.112:0/2344950333 wait complete. 2024-04-25T16:37:53.703 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:53 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/143375048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:53.703 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:53 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1187742064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:37:53.703 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:53 smithi112 ceph-mon[29122]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T16:37:53.703 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:53 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/985686474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:37:53.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 -- 172.21.15.112:0/389102664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4103c40 msgr2=0x7f60c4104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 --2- 172.21.15.112:0/389102664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4103c40 0x7f60c4104050 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009900 tx=0x7f60b402a410 comp rx=0 tx=0).stop 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 -- 172.21.15.112:0/389102664 shutdown_connections 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 --2- 172.21.15.112:0/389102664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60c4108940 0x7f60c410ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 --2- 172.21.15.112:0/389102664 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4103c40 0x7f60c4104050 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 --2- 172.21.15.112:0/389102664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60c4102a00 0x7f60c4103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 -- 172.21.15.112:0/389102664 >> 172.21.15.112:0/389102664 conn(0x7f60c40fdbf0 msgr2=0x7f60c41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:53.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 -- 172.21.15.112:0/389102664 shutdown_connections 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 -- 172.21.15.112:0/389102664 wait complete. 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 Processor -- start 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.813+0000 7f60cb469640 1 -- start start 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60c4102a00 0x7f60c410d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60c4103c40 0x7f60c410b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 0x7f60c410beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f60c410f630 con 0x7f60c4108940 2024-04-25T16:37:53.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60c410f4b0 con 0x7f60c4103c40 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f60c410f7b0 con 0x7f60c4102a00 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 0x7f60c410beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 0x7f60c410beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49134/0 (socket says 172.21.15.112:49134) 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 -- 172.21.15.112:0/3145174327 learned_addr learned my addr 172.21.15.112:0/3145174327 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9205640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60c4102a00 0x7f60c410d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c8a04640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60c4103c40 0x7f60c410b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 -- 172.21.15.112:0/3145174327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60c4102a00 msgr2=0x7f60c410d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60c4102a00 0x7f60c410d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 -- 172.21.15.112:0/3145174327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60c4103c40 msgr2=0x7f60c410b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60c4103c40 0x7f60c410b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 -- 172.21.15.112:0/3145174327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60c410c5e0 con 0x7f60c4108940 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60c9a06640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 0x7f60c410beb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00eb10 tx=0x7f60bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc004b20 con 0x7f60c4108940 2024-04-25T16:37:53.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60bc004cc0 con 0x7f60c4108940 2024-04-25T16:37:53.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c41a4290 con 0x7f60c4108940 2024-04-25T16:37:53.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc010670 con 0x7f60c4108940 2024-04-25T16:37:53.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.814+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60c41a4780 con 0x7f60c4108940 2024-04-25T16:37:53.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.815+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60c4107750 con 0x7f60c4108940 2024-04-25T16:37:53.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.817+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f60bc005000 con 0x7f60c4108940 2024-04-25T16:37:53.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.817+0000 7f60ba7fc640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f60b00afb70 0x7f60b00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:53.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.817+0000 7f60c9205640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f60b00afb70 0x7f60b00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:53.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.817+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f60bc0df400 con 0x7f60c4108940 2024-04-25T16:37:53.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.818+0000 7f60c9205640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f60b00afb70 0x7f60b00b2080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005d00 tx=0x7f60c0005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:53.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:53.821+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60bc053ff0 con 0x7f60c4108940 2024-04-25T16:37:53.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:53 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/143375048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:53.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:53 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1187742064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:37:53.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:53 smithi133 ceph-mon[32691]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T16:37:53.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:53 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/985686474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:37:54.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:53 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/143375048' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:54.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:53 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1187742064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:37:54.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:53 smithi116 ceph-mon[32712]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T16:37:54.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:53 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/985686474' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:37:54.033 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.033+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60c402db50 con 0x7f60c4108940 2024-04-25T16:37:54.034 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.034+0000 7f60ba7fc640 1 -- 172.21.15.112:0/3145174327 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11878 (secure 0 0 0) 0x7f60bc0a7fc0 con 0x7f60c4108940 2024-04-25T16:37:54.034 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:54.034 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:36:31.235238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24514,24524,24560]},"id":1}]} 2024-04-25T16:37:54.034 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:37:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f60b00afb70 msgr2=0x7f60b00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f60b00afb70 0x7f60b00b2080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f60c0005d00 tx=0x7f60c0005c50 comp rx=0 tx=0).stop 2024-04-25T16:37:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 msgr2=0x7f60c410beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 0x7f60c410beb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00eb10 tx=0x7f60bc00c2d0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 shutdown_connections 2024-04-25T16:37:54.036 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f60c4108940 0x7f60c410beb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60c4103c40 0x7f60c410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f60b00afb70 0x7f60b00b2080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 --2- 172.21.15.112:0/3145174327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60c4102a00 0x7f60c410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 >> 172.21.15.112:0/3145174327 conn(0x7f60c40fdbf0 msgr2=0x7f60c4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 shutdown_connections 2024-04-25T16:37:54.037 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.036+0000 7f60cb469640 1 -- 172.21.15.112:0/3145174327 wait complete. 2024-04-25T16:37:54.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 -- 172.21.15.112:0/3073404117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8481098b0 msgr2=0x7fe84810bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 --2- 172.21.15.112:0/3073404117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8481098b0 0x7fe84810bcd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe840009ca0 tx=0x7fe84002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 -- 172.21.15.112:0/3073404117 shutdown_connections 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 --2- 172.21.15.112:0/3073404117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8481098b0 0x7fe84810bcd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 --2- 172.21.15.112:0/3073404117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe848106f50 0x7fe848109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 --2- 172.21.15.112:0/3073404117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8481045f0 0x7fe848106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 -- 172.21.15.112:0/3073404117 >> 172.21.15.112:0/3073404117 conn(0x7fe8480fdbf0 msgr2=0x7fe8481000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 -- 172.21.15.112:0/3073404117 shutdown_connections 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 -- 172.21.15.112:0/3073404117 wait complete. 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.155+0000 7fe84eb84640 1 Processor -- start 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 -- start start 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8481045f0 0x7fe84819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 0x7fe84819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8481098b0 0x7fe8481a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe84810f630 con 0x7fe848106f50 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe84810f4b0 con 0x7fe8481098b0 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe84810f7b0 con 0x7fe8481045f0 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 0x7fe84819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 0x7fe84819a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49158/0 (socket says 172.21.15.112:49158) 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 -- 172.21.15.112:0/2182882816 learned_addr learned my addr 172.21.15.112:0/2182882816 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 -- 172.21.15.112:0/2182882816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8481045f0 msgr2=0x7fe84819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8481045f0 0x7fe84819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 -- 172.21.15.112:0/2182882816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8481098b0 msgr2=0x7fe8481a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8481098b0 0x7fe8481a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 -- 172.21.15.112:0/2182882816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8481a1c30 con 0x7fe848106f50 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe847fff640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 0x7fe84819a6e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c920 tx=0x7fe83400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834007c70 con 0x7fe848106f50 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe834007e10 con 0x7fe848106f50 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8481a1f20 con 0x7fe848106f50 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.156+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe834018820 con 0x7fe848106f50 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.157+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8481a23b0 con 0x7fe848106f50 2024-04-25T16:37:54.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.157+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8480ff710 con 0x7fe848106f50 2024-04-25T16:37:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.159+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fe83400f450 con 0x7fe848106f50 2024-04-25T16:37:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.159+0000 7fe845ffb640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe8380afaf0 0x7fe8380b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.159+0000 7fe84c920640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe8380afaf0 0x7fe8380b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.159+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fe8340dee50 con 0x7fe848106f50 2024-04-25T16:37:54.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.160+0000 7fe84c920640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe8380afaf0 0x7fe8380b2000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe83c005fd0 tx=0x7fe83c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.164+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe834014030 con 0x7fe848106f50 2024-04-25T16:37:54.342 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.343+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8480638a0 con 0x7fe848106f50 2024-04-25T16:37:54.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.344+0000 7fe845ffb640 1 -- 172.21.15.112:0/2182882816 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22824 (secure 0 0 0) 0x7fe8340a7a10 con 0x7fe848106f50 2024-04-25T16:37:54.344 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:54.344 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":77,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:37:16.199786+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:37:54.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe8380afaf0 msgr2=0x7fe8380b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe8380afaf0 0x7fe8380b2000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe83c005fd0 tx=0x7fe83c005850 comp rx=0 tx=0).stop 2024-04-25T16:37:54.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 msgr2=0x7fe84819a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 0x7fe84819a6e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c920 tx=0x7fe83400cde0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 shutdown_connections 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8481098b0 0x7fe8481a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe848106f50 0x7fe84819a6e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe8380afaf0 0x7fe8380b2000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 --2- 172.21.15.112:0/2182882816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8481045f0 0x7fe84819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 >> 172.21.15.112:0/2182882816 conn(0x7fe8480fdbf0 msgr2=0x7fe8481054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 shutdown_connections 2024-04-25T16:37:54.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.346+0000 7fe84eb84640 1 -- 172.21.15.112:0/2182882816 wait complete. 2024-04-25T16:37:54.358 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:37:54.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 -- 172.21.15.112:0/1243850168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 msgr2=0x7ff9c810ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 --2- 172.21.15.112:0/1243850168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c810ad20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009ca0 tx=0x7ff9c002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 -- 172.21.15.112:0/1243850168 shutdown_connections 2024-04-25T16:37:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 --2- 172.21.15.112:0/1243850168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9c8103c00 0x7ff9c8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 --2- 172.21.15.112:0/1243850168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c810ad20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 --2- 172.21.15.112:0/1243850168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 0x7ff9c81036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 -- 172.21.15.112:0/1243850168 >> 172.21.15.112:0/1243850168 conn(0x7ff9c80fdbd0 msgr2=0x7ff9c8100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 -- 172.21.15.112:0/1243850168 shutdown_connections 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 -- 172.21.15.112:0/1243850168 wait complete. 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.467+0000 7ff9cd584640 1 Processor -- start 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 -- start start 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 0x7ff9c806a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9c8103c00 0x7ff9c819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c81a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c810e4d0 con 0x7ff9c8108900 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c810e350 con 0x7ff9c81029c0 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9cd584640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c810e650 con 0x7ff9c8103c00 2024-04-25T16:37:54.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c81a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c81a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49180/0 (socket says 172.21.15.112:49180) 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 -- 172.21.15.112:0/806405236 learned_addr learned my addr 172.21.15.112:0/806405236 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c67fc640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9c8103c00 0x7ff9c819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 -- 172.21.15.112:0/806405236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9c8103c00 msgr2=0x7ff9c819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9c8103c00 0x7ff9c819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 -- 172.21.15.112:0/806405236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 msgr2=0x7ff9c806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c6ffd640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 0x7ff9c806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 0x7ff9c806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 -- 172.21.15.112:0/806405236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c81a1cc0 con 0x7ff9c8108900 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c6ffd640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 0x7ff9c806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9c77fe640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c81a14a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00097b0 tx=0x7ff9c0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.468+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0004240 con 0x7ff9c8108900 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.469+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00317c0 con 0x7ff9c8108900 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.469+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c81a1f50 con 0x7ff9c8108900 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.469+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c002fde0 con 0x7ff9c8108900 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.469+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c81a23b0 con 0x7ff9c8108900 2024-04-25T16:37:54.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.469+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c806afa0 con 0x7ff9c8108900 2024-04-25T16:37:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.471+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7ff9c00043e0 con 0x7ff9c8108900 2024-04-25T16:37:54.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.471+0000 7ff9affff640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff9b00afaf0 0x7ff9b00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.471+0000 7ff9c6ffd640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff9b00afaf0 0x7ff9b00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.472+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff9c0100900 con 0x7ff9c8108900 2024-04-25T16:37:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.472+0000 7ff9c6ffd640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff9b00afaf0 0x7ff9b00b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9c81035c0 tx=0x7ff9b8005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.475+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff9c00c94c0 con 0x7ff9c8108900 2024-04-25T16:37:54.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.690+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9c802dc10 con 0x7ff9c8108900 2024-04-25T16:37:54.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.691+0000 7ff9affff640 1 -- 172.21.15.112:0/806405236 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11878 (secure 0 0 0) 0x7ff9c00c8dc0 con 0x7ff9c8108900 2024-04-25T16:37:54.690 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:37:54.690 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:36:31.235238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24514,24524,24560]},"id":1}]} 2024-04-25T16:37:54.690 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff9b00afaf0 msgr2=0x7ff9b00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff9b00afaf0 0x7ff9b00b2000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9c81035c0 tx=0x7ff9b8005d30 comp rx=0 tx=0).stop 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 msgr2=0x7ff9c81a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c81a14a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00097b0 tx=0x7ff9c0003c20 comp rx=0 tx=0).stop 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 shutdown_connections 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff9b00afaf0 0x7ff9b00b2000 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9c8103c00 0x7ff9c819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff9c8108900 0x7ff9c81a14a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 --2- 172.21.15.112:0/806405236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9c81029c0 0x7ff9c806a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 >> 172.21.15.112:0/806405236 conn(0x7ff9c80fdbd0 msgr2=0x7ff9c81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 shutdown_connections 2024-04-25T16:37:54.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.693+0000 7ff9cd584640 1 -- 172.21.15.112:0/806405236 wait complete. 2024-04-25T16:37:54.702 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:54 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2344950333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:37:54.702 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:54 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3145174327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:54.702 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:54 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2182882816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:54.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T16:36:26.563199+0000', 'modified': '2024-04-25T16:36:31.235238+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24560, 'mds_1': 24524, 'mds_2': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/2304762610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2304762610}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2304762610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6837/3840802165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 3840802165}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3840802165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.116:6835/4202555064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 4202555064}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 4202555064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6839/99930432', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 99930432}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 99930432}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.116:6837/797562383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 797562383}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 797562383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6839/1016899540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6838', 'nonce': 1016899540}, {'type': 'v1', 'addr': '172.21.15.116:6839', 'nonce': 1016899540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24560, 'qdb_cluster': [24514, 24524, 24560]} 2024-04-25T16:37:54.704 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T16:37:54.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T16:37:54.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.813+0000 7f2789413640 1 -- 172.21.15.112:0/1382818398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 msgr2=0x7f278410bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.813+0000 7f2789413640 1 --2- 172.21.15.112:0/1382818398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f278410bcb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f277c009ca0 tx=0x7f277c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.813+0000 7f2789413640 1 -- 172.21.15.112:0/1382818398 shutdown_connections 2024-04-25T16:37:54.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.813+0000 7f2789413640 1 --2- 172.21.15.112:0/1382818398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2784106f30 0x7f2784109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.813+0000 7f2789413640 1 --2- 172.21.15.112:0/1382818398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f278410bcb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.813+0000 7f2789413640 1 --2- 172.21.15.112:0/1382818398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27841045d0 0x7f27841069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 -- 172.21.15.112:0/1382818398 >> 172.21.15.112:0/1382818398 conn(0x7f27840fdbd0 msgr2=0x7f2784100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 -- 172.21.15.112:0/1382818398 shutdown_connections 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 -- 172.21.15.112:0/1382818398 wait complete. 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 Processor -- start 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 -- start start 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27841045d0 0x7f278419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.814+0000 7f2789413640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2784106f30 0x7f278419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2789413640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f27841a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2789413640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f278410f610 con 0x7f2784109890 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2789413640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f278410f490 con 0x7f2784106f30 2024-04-25T16:37:54.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2789413640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f278410f790 con 0x7f27841045d0 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f27841a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f27841a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49188/0 (socket says 172.21.15.112:49188) 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 -- 172.21.15.112:0/3916859557 learned_addr learned my addr 172.21.15.112:0/3916859557 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2782ffd640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27841045d0 0x7f278419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27827fc640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2784106f30 0x7f278419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 -- 172.21.15.112:0/3916859557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27841045d0 msgr2=0x7f278419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27841045d0 0x7f278419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 -- 172.21.15.112:0/3916859557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2784106f30 msgr2=0x7f278419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2784106f30 0x7f278419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 -- 172.21.15.112:0/3916859557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27841a1cd0 con 0x7f2784109890 2024-04-25T16:37:54.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f27837fe640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f27841a1710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f277c033040 tx=0x7f277c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f276bfff640 1 -- 172.21.15.112:0/3916859557 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c003c70 con 0x7f2784109890 2024-04-25T16:37:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841a1f60 con 0x7f2784109890 2024-04-25T16:37:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f276bfff640 1 -- 172.21.15.112:0/3916859557 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c003e10 con 0x7f2784109890 2024-04-25T16:37:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f276bfff640 1 -- 172.21.15.112:0/3916859557 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c02fb70 con 0x7f2784109890 2024-04-25T16:37:54.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.815+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841a23c0 con 0x7f2784109890 2024-04-25T16:37:54.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f276bfff640 1 -- 172.21.15.112:0/3916859557 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f277c02fd10 con 0x7f2784109890 2024-04-25T16:37:54.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 Processor -- start 2024-04-25T16:37:54.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 -- start start 2024-04-25T16:37:54.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 0x55b90deca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db003b0 0x55b90deca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db08590 0x55b90ded17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b90de2f740 con 0x55b90db29270 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b90ddcb3a0 con 0x55b90db003b0 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f279039d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b90de2b700 con 0x55b90db08590 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f27827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 0x55b90deca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f27827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 0x55b90deca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49198/0 (socket says 172.21.15.112:49198) 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.818+0000 7f27827fc640 1 -- 172.21.15.112:0/3442248971 learned_addr learned my addr 172.21.15.112:0/3442248971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f2782ffd640 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db003b0 0x55b90deca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f276bfff640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f276c0afaf0 0x7f276c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27827fc640 1 -- 172.21.15.112:0/3442248971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db08590 msgr2=0x55b90ded17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27827fc640 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db08590 0x55b90ded17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27827fc640 1 -- 172.21.15.112:0/3442248971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db003b0 msgr2=0x55b90deca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27827fc640 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db003b0 0x55b90deca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27827fc640 1 -- 172.21.15.112:0/3442248971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90dde7750 con 0x55b90db29270 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27837fe640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f276c0afaf0 0x7f276c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27827fc640 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 0x55b90deca370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f277800b790 tx=0x7f277800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a308f8c9b7e9bb2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f276bfff640 1 -- 172.21.15.112:0/3916859557 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f277c100160 con 0x7f2784109890 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27697fa640 1 -- 172.21.15.112:0/3442248971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277800bea0 con 0x55b90db29270 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27697fa640 1 -- 172.21.15.112:0/3442248971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778016030 con 0x55b90db29270 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.819+0000 7f27697fa640 1 -- 172.21.15.112:0/3442248971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778004d90 con 0x55b90db29270 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 -- 172.21.15.112:0/3442248971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 msgr2=0x55b90deca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 0x55b90deca370 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f277800b790 tx=0x7f277800bc50 comp rx=0 tx=0).stop 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f27837fe640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f276c0afaf0 0x7f276c0b2000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f277c0c8320 tx=0x7f277c045f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 -- 172.21.15.112:0/3442248971 shutdown_connections 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db08590 0x55b90ded17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db003b0 0x55b90deca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 --2- 172.21.15.112:0/3442248971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db29270 0x55b90deca370 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 -- 172.21.15.112:0/3442248971 >> 172.21.15.112:0/3442248971 conn(0x55b90db1d3c0 msgr2=0x55b90dec5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 -- 172.21.15.112:0/3442248971 shutdown_connections 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 -- 172.21.15.112:0/3442248971 wait complete. 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 -- start start 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.820+0000 7f279039d740 1 Processor -- start 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db29270 0x55b90dedb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b90ded21e0 con 0x55b90db003b0 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b90ddb6fe0 con 0x55b90db08590 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b90ded2360 con 0x55b90db29270 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 10 client.0 _ll_register_callbacks cb 0x55b90ddcd150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49202/0 (socket says 172.21.15.112:49202) 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 -- 172.21.15.112:0/1017685540 learned_addr learned my addr 172.21.15.112:0/1017685540 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f2782ffd640 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 -- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db29270 msgr2=0x55b90dedb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db29270 0x55b90dedb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 -- 172.21.15.112:0/1017685540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 msgr2=0x55b90ded4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 -- 172.21.15.112:0/1017685540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90db412d0 con 0x55b90db003b0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f27827fc640 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded41d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2778004330 tx=0x7f27780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3565110da6f19078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f2766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f2766ff5640 1 -- 172.21.15.112:0/1017685540 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277801b670 con 0x55b90db003b0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f2766ff5640 1 -- 172.21.15.112:0/1017685540 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f277801b810 con 0x55b90db003b0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2778016030 con 0x55b90db003b0 2024-04-25T16:37:54.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f2766ff5640 1 -- 172.21.15.112:0/1017685540 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f277801ba10 con 0x55b90db003b0 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 10 client.15066 fetch_fsmap learned FSMap version 21 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.821+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b90dc0b6b0 con 0x55b90db003b0 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f2766ff5640 1 -- 172.21.15.112:0/1017685540 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778016030 con 0x55b90db003b0 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f2766ff5640 1 -- 172.21.15.112:0/1017685540 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f2778010c80 con 0x55b90db003b0 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f279039d740 10 client.15066 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f279039d740 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f279039d740 20 client.15066 populate_metadata read hostname 'smithi112' 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f279039d740 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90dae7e40 0x55b90dee1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f279039d740 4 client.15066 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 --> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b90de3d350 con 0x55b90dae7e40 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.822+0000 7f2782ffd640 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90dae7e40 0x55b90dee1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.823+0000 7f2766ff5640 10 client.15066 ms_handle_connect on v2:172.21.15.133:6838/99930432 2024-04-25T16:37:54.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.823+0000 7f2782ffd640 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90dae7e40 0x55b90dee1cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d54c9a1fdb1e738 server_cookie=e4e2b219aaae44c9 in_seq=0 out_seq=0 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.823+0000 7f2766ff5640 1 -- 172.21.15.112:0/1017685540 <== mds.2 v2:172.21.15.133:6838/99930432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b90de3d350 con 0x55b90dae7e40 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.823+0000 7f2766ff5640 10 client.15066 handle_command_reply: tid=0 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.823+0000 7f279039d740 1 client.15066 shutdown 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 msgr2=0x55b90ded41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded41d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2778004330 tx=0x7f27780046c0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 shutdown_connections 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90dae7e40 0x55b90dee1cc0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90db29270 0x55b90dedb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f2766ff5640 0 client.15066 ms_handle_reset on v2:172.21.15.133:6838/99930432 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 --2- 172.21.15.112:0/1017685540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded41d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 >> 172.21.15.112:0/1017685540 conn(0x55b90db1d3c0 msgr2=0x55b90dec4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 shutdown_connections 2024-04-25T16:37:54.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 1 -- 172.21.15.112:0/1017685540 wait complete. 2024-04-25T16:37:54.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.824+0000 7f279039d740 20 client.15066 trim_cache size 0 max 16384 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 Processor -- start 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 -- start start 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90decd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90dec56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90dec5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b90dee0000 con 0x55b90db003b0 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b90dee06d0 con 0x55b90db08590 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f279039d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b90dee0da0 con 0x55b90dae7e40 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90dec56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90dec56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49206/0 (socket says 172.21.15.112:49206) 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 -- 172.21.15.112:0/1169258636 learned_addr learned my addr 172.21.15.112:0/1169258636 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f27827fc640 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90decd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f27827fc640 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90dec5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 -- 172.21.15.112:0/1169258636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 msgr2=0x55b90decd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90decd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 -- 172.21.15.112:0/1169258636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 msgr2=0x55b90dec5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90dec5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f2782ffd640 1 -- 172.21.15.112:0/1169258636 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90decf050 con 0x55b90db003b0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.842+0000 7f27827fc640 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90decd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f2782ffd640 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90dec56b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2774000c00 tx=0x7f27740118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbf04737881234c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f2768ff9640 1 -- 172.21.15.112:0/1169258636 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774021030 con 0x55b90db003b0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f2768ff9640 1 -- 172.21.15.112:0/1169258636 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277401da50 con 0x55b90db003b0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f2768ff9640 1 -- 172.21.15.112:0/1169258636 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2774004a00 con 0x55b90db003b0 2024-04-25T16:37:54.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f279039d740 1 -- 172.21.15.112:0/1169258636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 msgr2=0x55b90dec56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f279039d740 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90dec56b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2774000c00 tx=0x7f27740118d0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.843+0000 7f279039d740 1 -- 172.21.15.112:0/1169258636 shutdown_connections 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90dec5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90dec56b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 --2- 172.21.15.112:0/1169258636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90decd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- 172.21.15.112:0/1169258636 >> 172.21.15.112:0/1169258636 conn(0x55b90db1d3c0 msgr2=0x55b90dedf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- 172.21.15.112:0/1169258636 shutdown_connections 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- 172.21.15.112:0/1169258636 wait complete. 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- start start 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 Processor -- start 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90ded62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b90dee1b40 con 0x55b90db003b0 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b90dee1470 con 0x55b90db08590 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b90dee2210 con 0x55b90dae7e40 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f279039d740 10 client.0 _ll_register_callbacks cb 0x7f277800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49220/0 (socket says 172.21.15.112:49220) 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 -- 172.21.15.112:0/3877562117 learned_addr learned my addr 172.21.15.112:0/3877562117 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f27827fc640 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90ded62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 -- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 msgr2=0x55b90ded62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90ded62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 -- 172.21.15.112:0/3877562117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 msgr2=0x55b90ded9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.844+0000 7f2782ffd640 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2782ffd640 1 -- 172.21.15.112:0/3877562117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90ded9c20 con 0x55b90db003b0 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2782ffd640 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded67e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f277400b010 tx=0x7f277401d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fa6ea989071292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2766ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2766ff5640 1 -- 172.21.15.112:0/3877562117 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27740046c0 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2766ff5640 1 -- 172.21.15.112:0/3877562117 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2774004860 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277401da50 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2766ff5640 1 -- 172.21.15.112:0/3877562117 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2774035070 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 10 client.15072 fetch_fsmap learned FSMap version 21 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55b90dedaed0 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2766ff5640 1 -- 172.21.15.112:0/3877562117 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277401da50 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f2766ff5640 1 -- 172.21.15.112:0/3877562117 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f277402fb80 con 0x55b90db003b0 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 10 client.15072 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 20 client.15072 populate_metadata read hostname 'smithi112' 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90db29270 0x55b90df4bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 4 client.15072 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.845+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 --> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] -- command(tid 0: {"prefix": "status"}) -- 0x55b90de371c0 con 0x55b90db29270 2024-04-25T16:37:54.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f27827fc640 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90db29270 0x55b90df4bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f2766ff5640 10 client.15072 ms_handle_connect on v2:172.21.15.133:6838/99930432 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f27827fc640 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90db29270 0x55b90df4bb30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f83a40cd9016aeb server_cookie=34f1470d451b2c02 in_seq=0 out_seq=0 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f2766ff5640 1 -- 172.21.15.112:0/3877562117 <== mds.2 v2:172.21.15.133:6838/99930432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b90de371c0 con 0x55b90db29270 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f2766ff5640 10 client.15072 handle_command_reply: tid=0 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 client.15072 shutdown 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 msgr2=0x55b90ded67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded67e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f277400b010 tx=0x7f277401d7d0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 shutdown_connections 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x55b90db29270 0x55b90df4bb30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b90db08590 0x55b90ded9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b90db003b0 0x55b90ded67e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f2766ff5640 0 client.15072 ms_handle_reset on v2:172.21.15.133:6838/99930432 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 --2- 172.21.15.112:0/3877562117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b90dae7e40 0x55b90ded62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 >> 172.21.15.112:0/3877562117 conn(0x55b90db1d3c0 msgr2=0x55b90decd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 shutdown_connections 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.846+0000 7f279039d740 1 -- 172.21.15.112:0/3877562117 wait complete. 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f279039d740 20 client.15072 trim_cache size 0 max 16384 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24514, 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T16:37:54.852 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 84.621928320999999, 2024-04-25T16:37:54.853 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-04-25T16:37:54.853 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 77, 2024-04-25T16:37:54.853 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T16:37:54.853 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 103.00763268 2024-04-25T16:37:54.853 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f276c0afaf0 msgr2=0x7f276c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f276c0afaf0 0x7f276c0b2000 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f277c0c8320 tx=0x7f277c045f90 comp rx=0 tx=0).stop 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 msgr2=0x7f27841a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f27841a1710 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f277c033040 tx=0x7f277c0319f0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 shutdown_connections 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2784109890 0x7f27841a1710 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2784106f30 0x7f278419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f276c0afaf0 0x7f276c0b2000 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 --2- 172.21.15.112:0/3916859557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27841045d0 0x7f278419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 >> 172.21.15.112:0/3916859557 conn(0x7f27840fdbd0 msgr2=0x7f2784106590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 shutdown_connections 2024-04-25T16:37:54.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.847+0000 7f2789413640 1 -- 172.21.15.112:0/3916859557 wait complete. 2024-04-25T16:37:54.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:54 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2344950333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:37:54.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:54 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3145174327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:54.855 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:54 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2182882816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:54.857 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T16:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 -- 172.21.15.112:0/561616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168106a20 msgr2=0x7ff168108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 --2- 172.21.15.112:0/561616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168106a20 0x7ff168108f00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff150009900 tx=0x7ff15002a410 comp rx=0 tx=0).stop 2024-04-25T16:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 -- 172.21.15.112:0/561616812 shutdown_connections 2024-04-25T16:37:54.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 --2- 172.21.15.112:0/561616812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff168109590 0x7ff16810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 --2- 172.21.15.112:0/561616812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168106a20 0x7ff168108f00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 --2- 172.21.15.112:0/561616812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff168104030 0x7ff168106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 -- 172.21.15.112:0/561616812 >> 172.21.15.112:0/561616812 conn(0x7ff1680fd9f0 msgr2=0x7ff1680ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 -- 172.21.15.112:0/561616812 shutdown_connections 2024-04-25T16:37:54.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.966+0000 7ff167fff640 1 -- 172.21.15.112:0/561616812 wait complete. 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 Processor -- start 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 -- start start 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 0x7ff16819a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff168106a20 0x7ff16819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff168109590 0x7ff1681a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810f3e0 con 0x7ff168104030 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810f260 con 0x7ff168109590 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.967+0000 7ff167fff640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff16810f560 con 0x7ff168106a20 2024-04-25T16:37:54.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 0x7ff16819a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 0x7ff16819a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49230/0 (socket says 172.21.15.112:49230) 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 -- 172.21.15.112:0/2917438611 learned_addr learned my addr 172.21.15.112:0/2917438611 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff168109590 0x7ff1681a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff168106a20 0x7ff16819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 -- 172.21.15.112:0/2917438611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff168106a20 msgr2=0x7ff16819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff168106a20 0x7ff16819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 -- 172.21.15.112:0/2917438611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff168109590 msgr2=0x7ff1681a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff168109590 0x7ff1681a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 -- 172.21.15.112:0/2917438611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1681a1b50 con 0x7ff168104030 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff166ffd640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 0x7ff16819a010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff15800d770 tx=0x7ff15800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff164ff9640 1 -- 172.21.15.112:0/2917438611 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158017070 con 0x7ff168104030 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff164ff9640 1 -- 172.21.15.112:0/2917438611 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff158009c30 con 0x7ff168104030 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1681a1e40 con 0x7ff168104030 2024-04-25T16:37:54.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff164ff9640 1 -- 172.21.15.112:0/2917438611 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158003cc0 con 0x7ff168104030 2024-04-25T16:37:54.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.968+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a22a0 con 0x7ff168104030 2024-04-25T16:37:54.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 Processor -- start 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff164ff9640 1 -- 172.21.15.112:0/2917438611 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7ff158003e60 con 0x7ff168104030 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 -- start start 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x55864768f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864768fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c9590 0x558647696df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586475f0740 con 0x5586472ea270 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55864758c3a0 con 0x5586472c13b0 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff17478b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586475ec700 con 0x5586472c9590 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x55864768f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x55864768f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49232/0 (socket says 172.21.15.112:49232) 2024-04-25T16:37:54.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff1677fe640 1 -- 172.21.15.112:0/1771500631 learned_addr learned my addr 172.21.15.112:0/1771500631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff166ffd640 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864768fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.971+0000 7ff15ffff640 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c9590 0x558647696df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff1677fe640 1 -- 172.21.15.112:0/1771500631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c9590 msgr2=0x558647696df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff1677fe640 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c9590 0x558647696df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff1677fe640 1 -- 172.21.15.112:0/1771500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 msgr2=0x55864768fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff1677fe640 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864768fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff1677fe640 1 -- 172.21.15.112:0/1771500631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586475a8750 con 0x5586472ea270 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff15ffff640 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c9590 0x558647696df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff164ff9640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1540afae0 0x7ff1540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff166ffd640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1540afae0 0x7ff1540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff1677fe640 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x55864768f970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff160009fd0 tx=0x7ff1600118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0a5fba67fdd58e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff164ff9640 1 -- 172.21.15.112:0/2917438611 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff1580de520 con 0x7ff168104030 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff15d7fa640 1 -- 172.21.15.112:0/1771500631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160004d20 con 0x5586472ea270 2024-04-25T16:37:54.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff15d7fa640 1 -- 172.21.15.112:0/1771500631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff160004050 con 0x5586472ea270 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.972+0000 7ff15d7fa640 1 -- 172.21.15.112:0/1771500631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160005280 con 0x5586472ea270 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 -- 172.21.15.112:0/1771500631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 msgr2=0x55864768f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x55864768f970 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff160009fd0 tx=0x7ff1600118d0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff166ffd640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1540afae0 0x7ff1540b1ff0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558647690d50 tx=0x7ff1580e4060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 -- 172.21.15.112:0/1771500631 shutdown_connections 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c9590 0x558647696df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864768fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 --2- 172.21.15.112:0/1771500631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x55864768f970 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 -- 172.21.15.112:0/1771500631 >> 172.21.15.112:0/1771500631 conn(0x5586472de3c0 msgr2=0x558647686ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 -- 172.21.15.112:0/1771500631 shutdown_connections 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.973+0000 7ff17478b740 1 -- 172.21.15.112:0/1771500631 wait complete. 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 -- start start 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 Processor -- start 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c13b0 0x558647699880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647699dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x5586476a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558647697840 con 0x5586472ea270 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558647577fe0 con 0x5586472c9590 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586476979c0 con 0x5586472c13b0 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff17478b740 10 client.0 _ll_register_callbacks cb 0x55864758e150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x5586476a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x5586476a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49246/0 (socket says 172.21.15.112:49246) 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 -- 172.21.15.112:0/2987890580 learned_addr learned my addr 172.21.15.112:0/2987890580 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647699dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c13b0 0x558647699880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 -- 172.21.15.112:0/2987890580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c13b0 msgr2=0x558647699880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c13b0 0x558647699880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 -- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 msgr2=0x558647699dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647699dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.974+0000 7ff1677fe640 1 -- 172.21.15.112:0/2987890580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586473022d0 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x5586476a0d20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff160005b40 tx=0x7ff160005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d4a3001ce3def6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff14e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2987890580 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16002d060 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2987890580 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff160004a40 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff160004050 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2987890580 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff160004600 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 10 client.15084 fetch_fsmap learned FSMap version 21 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x5586473cc6b0 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2987890580 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff160004050 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2987890580 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff160005030 con 0x5586472ea270 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 10 client.15084 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 20 client.15084 populate_metadata read hostname 'smithi112' 2024-04-25T16:37:54.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x5586472a8e40 0x5586476a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 4 client.15084 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.975+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 --> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586475fe350 con 0x5586472a8e40 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.976+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x5586472a8e40 0x5586476a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.976+0000 7ff14e7fc640 10 client.15084 ms_handle_connect on v2:172.21.15.116:6836/797562383 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.976+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x5586472a8e40 0x5586476a73a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb1d2ba383c71b7a server_cookie=8674cce5a29f004d in_seq=0 out_seq=0 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2987890580 <== mds.1 v2:172.21.15.116:6836/797562383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5586475fe350 con 0x5586472a8e40 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff14e7fc640 10 client.15084 handle_command_reply: tid=0 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 client.15084 shutdown 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 msgr2=0x5586476a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x5586476a0d20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff160005b40 tx=0x7ff160005ab0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 shutdown_connections 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x5586472a8e40 0x5586476a73a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472ea270 0x5586476a0d20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647699dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 --2- 172.21.15.112:0/2987890580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472c13b0 0x558647699880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 >> 172.21.15.112:0/2987890580 conn(0x5586472de3c0 msgr2=0x558647685f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff14e7fc640 0 client.15084 ms_handle_reset on v2:172.21.15.116:6836/797562383 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 shutdown_connections 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.977+0000 7ff17478b740 1 -- 172.21.15.112:0/2987890580 wait complete. 2024-04-25T16:37:54.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.978+0000 7ff17478b740 20 client.15084 trim_cache size 0 max 16384 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 Processor -- start 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 -- start start 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 0x558647692f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647693ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x5586476941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586476a56e0 con 0x5586472c13b0 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5586476a5db0 con 0x5586472c9590 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff17478b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586476a6480 con 0x5586472ea270 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.997+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 0x558647692f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 0x558647692f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49256/0 (socket says 172.21.15.112:49256) 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 -- 172.21.15.112:0/1778871272 learned_addr learned my addr 172.21.15.112:0/1778871272 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff15ffff640 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647693ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:54.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 -- 172.21.15.112:0/1778871272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 msgr2=0x5586476941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x5586476941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 -- 172.21.15.112:0/1778871272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 msgr2=0x558647693ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647693ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 -- 172.21.15.112:0/1778871272 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558647681140 con 0x5586472c13b0 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff15ffff640 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647693ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff1677fe640 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 0x558647692f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff160006010 tx=0x7ff160005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980e06de21943f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff15cff9640 1 -- 172.21.15.112:0/1778871272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16002d050 con 0x5586472c13b0 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff15cff9640 1 -- 172.21.15.112:0/1778871272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff160003a30 con 0x5586472c13b0 2024-04-25T16:37:54.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.998+0000 7ff15cff9640 1 -- 172.21.15.112:0/1778871272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16000fd90 con 0x5586472c13b0 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 -- 172.21.15.112:0/1778871272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 msgr2=0x558647692f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 0x558647692f80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff160006010 tx=0x7ff160005cf0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 -- 172.21.15.112:0/1778871272 shutdown_connections 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x5586476941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c9590 0x558647693ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 --2- 172.21.15.112:0/1778871272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c13b0 0x558647692f80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 -- 172.21.15.112:0/1778871272 >> 172.21.15.112:0/1778871272 conn(0x5586472de3c0 msgr2=0x558647699840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 -- 172.21.15.112:0/1778871272 shutdown_connections 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 -- 172.21.15.112:0/1778871272 wait complete. 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:54.999+0000 7ff17478b740 1 -- start start 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 Processor -- start 2024-04-25T16:37:55.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864769b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 0x55864769b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x55864770e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5586476a7220 con 0x5586472c9590 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5586476a6b50 con 0x5586472c13b0 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5586476a78f0 con 0x5586472ea270 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff17478b740 10 client.0 _ll_register_callbacks cb 0x7ff1580e1ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 0x55864769b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 0x55864769b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49264/0 (socket says 172.21.15.112:49264) 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 -- 172.21.15.112:0/2243205765 learned_addr learned my addr 172.21.15.112:0/2243205765 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x55864770e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864769b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 -- 172.21.15.112:0/2243205765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 msgr2=0x55864770e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x55864770e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 -- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 msgr2=0x55864769b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864769b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 -- 172.21.15.112:0/2243205765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586476a0460 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x55864770e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff15ffff640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 0x55864769b570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff15002a930 tx=0x7ff150005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a623a38646c5df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff14e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.000+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2243205765 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150035040 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2243205765 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7ff150045310 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586475f81c0 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2243205765 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff150053070 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 10 client.15090 fetch_fsmap learned FSMap version 21 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55864770f5f0 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2243205765 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5586475f81c0 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2243205765 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7ff150003aa0 con 0x5586472c9590 2024-04-25T16:37:55.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 10 client.15090 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 20 client.15090 populate_metadata read hostname 'smithi112' 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x55864727e280 0x558647712f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 4 client.15090 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 --> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] -- command(tid 0: {"prefix": "status"}) -- 0x5586475739a0 con 0x55864727e280 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.001+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x55864727e280 0x558647712f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff14e7fc640 10 client.15090 ms_handle_connect on v2:172.21.15.116:6836/797562383 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff1677fe640 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x55864727e280 0x558647712f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ef6641da874bc7f server_cookie=ea05e3a12932f1b9 in_seq=0 out_seq=0 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff14e7fc640 1 -- 172.21.15.112:0/2243205765 <== mds.1 v2:172.21.15.116:6836/797562383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5586475739a0 con 0x55864727e280 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff14e7fc640 10 client.15090 handle_command_reply: tid=0 2024-04-25T16:37:55.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 client.15090 shutdown 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 msgr2=0x55864769b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 0x55864769b570 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff15002a930 tx=0x7ff150005850 comp rx=0 tx=0).stop 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 shutdown_connections 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x55864727e280 0x558647712f60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5586472ea270 0x55864770e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff14e7fc640 0 client.15090 ms_handle_reset on v2:172.21.15.116:6836/797562383 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5586472c9590 0x55864769b570 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 --2- 172.21.15.112:0/2243205765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5586472c13b0 0x55864769b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 >> 172.21.15.112:0/2243205765 conn(0x5586472de3c0 msgr2=0x558647699840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 shutdown_connections 2024-04-25T16:37:55.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 1 -- 172.21.15.112:0/2243205765 wait complete. 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.002+0000 7ff17478b740 20 client.15090 trim_cache size 0 max 16384 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24524, 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 85.783692626999994, 2024-04-25T16:37:55.008 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-04-25T16:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 77, 2024-04-25T16:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T16:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 105.91816387199999 2024-04-25T16:37:55.009 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:37:55.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:54 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2344950333' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:37:55.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:54 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3145174327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:55.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:54 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2182882816' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:37:55.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1540afae0 msgr2=0x7ff1540b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1540afae0 0x7ff1540b1ff0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558647690d50 tx=0x7ff1580e4060 comp rx=0 tx=0).stop 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 msgr2=0x7ff16819a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 0x7ff16819a010 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff15800d770 tx=0x7ff15800dc30 comp rx=0 tx=0).stop 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 shutdown_connections 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7ff1540afae0 0x7ff1540b1ff0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff168106a20 0x7ff16819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff168109590 0x7ff1681a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 --2- 172.21.15.112:0/2917438611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff168104030 0x7ff16819a010 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 >> 172.21.15.112:0/2917438611 conn(0x7ff1680fd9f0 msgr2=0x7ff168106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 shutdown_connections 2024-04-25T16:37:55.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.003+0000 7ff167fff640 1 -- 172.21.15.112:0/2917438611 wait complete. 2024-04-25T16:37:55.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 -- 172.21.15.112:0/3949111566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 msgr2=0x7f72ec10be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 --2- 172.21.15.112:0/3949111566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec10be50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72e4009ca0 tx=0x7f72e402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 -- 172.21.15.112:0/3949111566 shutdown_connections 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 --2- 172.21.15.112:0/3949111566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec10be50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 --2- 172.21.15.112:0/3949111566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ec104d30 0x7f72ec105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 --2- 172.21.15.112:0/3949111566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec103af0 0x7f72ec1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 -- 172.21.15.112:0/3949111566 >> 172.21.15.112:0/3949111566 conn(0x7f72ec0fece0 msgr2=0x7f72ec101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 -- 172.21.15.112:0/3949111566 shutdown_connections 2024-04-25T16:37:55.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.123+0000 7f72f2796640 1 -- 172.21.15.112:0/3949111566 wait complete. 2024-04-25T16:37:55.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 Processor -- start 2024-04-25T16:37:55.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 -- start start 2024-04-25T16:37:55.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ec103af0 0x7f72ec19a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec104d30 0x7f72ec19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec10f620 con 0x7f72ec109a30 2024-04-25T16:37:55.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec10f4a0 con 0x7f72ec103af0 2024-04-25T16:37:55.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f2796640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec10f7a0 con 0x7f72ec104d30 2024-04-25T16:37:55.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f0d33640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f0d33640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec1a16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49288/0 (socket says 172.21.15.112:49288) 2024-04-25T16:37:55.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72f0d33640 1 -- 172.21.15.112:0/1884379098 learned_addr learned my addr 172.21.15.112:0/1884379098 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:55.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72ebfff640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ec103af0 0x7f72ec19a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.124+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec104d30 0x7f72ec19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f0d33640 1 -- 172.21.15.112:0/1884379098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec104d30 msgr2=0x7f72ec19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f0d33640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec104d30 0x7f72ec19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f0d33640 1 -- 172.21.15.112:0/1884379098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ec103af0 msgr2=0x7f72ec19a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f0d33640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ec103af0 0x7f72ec19a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f0d33640 1 -- 172.21.15.112:0/1884379098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec1a1d40 con 0x7f72ec109a30 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f0d33640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec1a16d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f72e40097b0 tx=0x7f72e4031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72e97fa640 1 -- 172.21.15.112:0/1884379098 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e4003bf0 con 0x7f72ec109a30 2024-04-25T16:37:55.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72e97fa640 1 -- 172.21.15.112:0/1884379098 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e4003d90 con 0x7f72ec109a30 2024-04-25T16:37:55.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec1a1fd0 con 0x7f72ec109a30 2024-04-25T16:37:55.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72e97fa640 1 -- 172.21.15.112:0/1884379098 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e402fc30 con 0x7f72ec109a30 2024-04-25T16:37:55.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.125+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec1a2430 con 0x7f72ec109a30 2024-04-25T16:37:55.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72e97fa640 1 -- 172.21.15.112:0/1884379098 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f72e4004040 con 0x7f72ec109a30 2024-04-25T16:37:55.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 Processor -- start 2024-04-25T16:37:55.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 -- start start 2024-04-25T16:37:55.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc930370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc9308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc9377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc895740 con 0x55f8dc58f270 2024-04-25T16:37:55.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc8313a0 con 0x55f8dc5663b0 2024-04-25T16:37:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f9649740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc891700 con 0x55f8dc56e590 2024-04-25T16:37:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc930370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc930370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49304/0 (socket says 172.21.15.112:49304) 2024-04-25T16:37:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2451098058 learned_addr learned my addr 172.21.15.112:0/2451098058 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72ebfff640 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc9308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc9377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72e97fa640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72d40afaf0 0x7f72d40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2451098058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 msgr2=0x55f8dc9377f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc9377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2451098058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 msgr2=0x55f8dc9308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc9308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2451098058 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8dc84d750 con 0x55f8dc58f270 2024-04-25T16:37:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72f0d33640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72d40afaf0 0x7f72d40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.128+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc930370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009e70 tx=0x7f72e0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baf5d0ffd50d725 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72d27fc640 1 -- 172.21.15.112:0/2451098058 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e001d650 con 0x55f8dc58f270 2024-04-25T16:37:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72e97fa640 1 -- 172.21.15.112:0/1884379098 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f72e41002d0 con 0x7f72ec109a30 2024-04-25T16:37:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72d27fc640 1 -- 172.21.15.112:0/2451098058 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e00108d0 con 0x55f8dc58f270 2024-04-25T16:37:55.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72d27fc640 1 -- 172.21.15.112:0/2451098058 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e000f8f0 con 0x55f8dc58f270 2024-04-25T16:37:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72f9649740 1 -- 172.21.15.112:0/2451098058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 msgr2=0x55f8dc930370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72f0d33640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72d40afaf0 0x7f72d40b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72e40ba240 tx=0x7f72e40972e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.129+0000 7f72f9649740 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc930370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009e70 tx=0x7f72e0009850 comp rx=0 tx=0).stop 2024-04-25T16:37:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- 172.21.15.112:0/2451098058 shutdown_connections 2024-04-25T16:37:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc9377f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc9308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 --2- 172.21.15.112:0/2451098058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc930370 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- 172.21.15.112:0/2451098058 >> 172.21.15.112:0/2451098058 conn(0x55f8dc5833c0 msgr2=0x55f8dc92bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- 172.21.15.112:0/2451098058 shutdown_connections 2024-04-25T16:37:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- 172.21.15.112:0/2451098058 wait complete. 2024-04-25T16:37:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- start start 2024-04-25T16:37:55.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 Processor -- start 2024-04-25T16:37:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc93a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc941750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc938240 con 0x55f8dc58f270 2024-04-25T16:37:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc81cfe0 con 0x55f8dc5663b0 2024-04-25T16:37:55.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.130+0000 7f72f9649740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc9383c0 con 0x55f8dc56e590 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72f9649740 10 client.0 _ll_register_callbacks cb 0x55f8dc833150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc941750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc941750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49310/0 (socket says 172.21.15.112:49310) 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 -- 172.21.15.112:0/1100865419 learned_addr learned my addr 172.21.15.112:0/1100865419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72ebfff640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc93a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 -- 172.21.15.112:0/1100865419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 msgr2=0x55f8dc93a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc93a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 -- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 msgr2=0x55f8dc93a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 -- 172.21.15.112:0/1100865419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8dc5a72d0 con 0x55f8dc58f270 2024-04-25T16:37:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc941750 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004600 tx=0x7f72e000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9b1eec31ac2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:37:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72cb7fe640 1 -- 172.21.15.112:0/1100865419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0026040 con 0x55f8dc58f270 2024-04-25T16:37:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72cb7fe640 1 -- 172.21.15.112:0/1100865419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f72e000fa80 con 0x55f8dc58f270 2024-04-25T16:37:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e00108d0 con 0x55f8dc58f270 2024-04-25T16:37:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72cb7fe640 1 -- 172.21.15.112:0/1100865419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f72e000fc80 con 0x55f8dc58f270 2024-04-25T16:37:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72f9649740 10 client.15102 fetch_fsmap learned FSMap version 21 2024-04-25T16:37:55.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.131+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f8dc6716b0 con 0x55f8dc58f270 2024-04-25T16:37:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72cb7fe640 1 -- 172.21.15.112:0/1100865419 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e00108d0 con 0x55f8dc58f270 2024-04-25T16:37:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72cb7fe640 1 -- 172.21.15.112:0/1100865419 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f72e002d430 con 0x55f8dc58f270 2024-04-25T16:37:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72f9649740 10 client.15102 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:37:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72f9649740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:37:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72f9649740 20 client.15102 populate_metadata read hostname 'smithi112' 2024-04-25T16:37:55.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72f9649740 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc54de40 0x55f8dc947e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72f9649740 4 client.15102 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:37:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8dc8a3350 con 0x55f8dc54de40 2024-04-25T16:37:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72ebfff640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc54de40 0x55f8dc947e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72cb7fe640 10 client.15102 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:37:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.132+0000 7f72ebfff640 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc54de40 0x55f8dc947e00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5092a96858dbec server_cookie=f20c27c22fdf8e7e in_seq=0 out_seq=0 2024-04-25T16:37:55.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72cb7fe640 1 -- 172.21.15.112:0/1100865419 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f8dc8a3350 con 0x55f8dc54de40 2024-04-25T16:37:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72cb7fe640 10 client.15102 handle_command_reply: tid=0 2024-04-25T16:37:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 client.15102 shutdown 2024-04-25T16:37:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 msgr2=0x55f8dc941750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc941750 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f72e0004600 tx=0x7f72e000f3c0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 shutdown_connections 2024-04-25T16:37:55.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc54de40 0x55f8dc947e00 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc56e590 0x55f8dc93a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc58f270 0x55f8dc941750 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 --2- 172.21.15.112:0/1100865419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72cb7fe640 0 client.15102 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.133+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 >> 172.21.15.112:0/1100865419 conn(0x55f8dc5833c0 msgr2=0x55f8dc92ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.134+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 shutdown_connections 2024-04-25T16:37:55.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.134+0000 7f72f9649740 1 -- 172.21.15.112:0/1100865419 wait complete. 2024-04-25T16:37:55.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.134+0000 7f72f9649740 20 client.15102 trim_cache size 0 max 16384 2024-04-25T16:37:55.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 Processor -- start 2024-04-25T16:37:55.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 -- start start 2024-04-25T16:37:55.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc9459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 0x55f8dc93a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc946140 con 0x55f8dc56e590 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc946810 con 0x55f8dc5663b0 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72f9649740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc946ee0 con 0x55f8dc54de40 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 0x55f8dc93a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 0x55f8dc93a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49320/0 (socket says 172.21.15.112:49320) 2024-04-25T16:37:55.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2374375206 learned_addr learned my addr 172.21.15.112:0/2374375206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc9459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.152+0000 7f72ebfff640 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2374375206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 msgr2=0x55f8dc9459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc9459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2374375206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 msgr2=0x55f8dc93a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72eb7fe640 1 -- 172.21.15.112:0/2374375206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8dc944590 con 0x55f8dc56e590 2024-04-25T16:37:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 0x55f8dc93a790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009860 tx=0x7f72e0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fb121108d5ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72d1ffb640 1 -- 172.21.15.112:0/2374375206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0021070 con 0x55f8dc56e590 2024-04-25T16:37:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72d1ffb640 1 -- 172.21.15.112:0/2374375206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e0013cc0 con 0x55f8dc56e590 2024-04-25T16:37:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.153+0000 7f72d1ffb640 1 -- 172.21.15.112:0/2374375206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e002d430 con 0x55f8dc56e590 2024-04-25T16:37:55.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 -- 172.21.15.112:0/2374375206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 msgr2=0x55f8dc93a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 0x55f8dc93a790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009860 tx=0x7f72e0005970 comp rx=0 tx=0).stop 2024-04-25T16:37:55.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 -- 172.21.15.112:0/2374375206 shutdown_connections 2024-04-25T16:37:55.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc56e590 0x55f8dc93a790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc5663b0 0x55f8dc93a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 --2- 172.21.15.112:0/2374375206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc9459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 -- 172.21.15.112:0/2374375206 >> 172.21.15.112:0/2374375206 conn(0x55f8dc5833c0 msgr2=0x55f8dc926a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 -- 172.21.15.112:0/2374375206 shutdown_connections 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 -- 172.21.15.112:0/2374375206 wait complete. 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 -- start start 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 Processor -- start 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc93bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.154+0000 7f72f9649740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 0x55f8dc93c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc56e590 0x55f8dc9aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc947c80 con 0x55f8dc5663b0 2024-04-25T16:37:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc9475b0 con 0x55f8dc56e590 2024-04-25T16:37:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8dc948350 con 0x55f8dc54de40 2024-04-25T16:37:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 10 client.0 _ll_register_callbacks cb 0x7f72dc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:37:55.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 0x55f8dc93c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 0x55f8dc93c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49328/0 (socket says 172.21.15.112:49328) 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 -- 172.21.15.112:0/2068142470 learned_addr learned my addr 172.21.15.112:0/2068142470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc56e590 0x55f8dc9aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc93bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 -- 172.21.15.112:0/2068142470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 msgr2=0x55f8dc93bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc93bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 -- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc56e590 msgr2=0x55f8dc9aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:37:55.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc56e590 0x55f8dc9aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 -- 172.21.15.112:0/2068142470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8dc93c6b0 con 0x55f8dc5663b0 2024-04-25T16:37:55.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72ebfff640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 0x55f8dc93c0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005fd0 tx=0x7f72dc012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6065b4d068ee5f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:55.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:37:55.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72cb7fe640 1 -- 172.21.15.112:0/2068142470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc01eaf0 con 0x55f8dc5663b0 2024-04-25T16:37:55.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72cb7fe640 1 -- 172.21.15.112:0/2068142470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f72dc01ec90 con 0x55f8dc5663b0 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e0013cc0 con 0x55f8dc5663b0 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72cb7fe640 1 -- 172.21.15.112:0/2068142470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f72dc005020 con 0x55f8dc5663b0 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 10 client.15108 fetch_fsmap learned FSMap version 21 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.155+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55f8dc9b02d0 con 0x55f8dc5663b0 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72cb7fe640 1 -- 172.21.15.112:0/2068142470 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e0013cc0 con 0x55f8dc5663b0 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72cb7fe640 1 -- 172.21.15.112:0/2068142470 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f72dc0200b0 con 0x55f8dc5663b0 2024-04-25T16:37:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72f9649740 10 client.15108 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72f9649740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72f9649740 20 client.15108 populate_metadata read hostname 'smithi112' 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72f9649740 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc58f270 0x55f8dc9b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72f9649740 4 client.15108 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "status"}) -- 0x55f8dc89d1c0 con 0x55f8dc58f270 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc58f270 0x55f8dc9b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72eb7fe640 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc58f270 0x55f8dc9b3c10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81edfa28d5cf5a06 server_cookie=36bfc89af7180943 in_seq=0 out_seq=0 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.156+0000 7f72cb7fe640 10 client.15108 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:37:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72cb7fe640 1 -- 172.21.15.112:0/2068142470 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f8dc89d1c0 con 0x55f8dc58f270 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72cb7fe640 10 client.15108 handle_command_reply: tid=0 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 client.15108 shutdown 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 msgr2=0x55f8dc93c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 0x55f8dc93c0c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f72dc005fd0 tx=0x7f72dc012b00 comp rx=0 tx=0).stop 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 shutdown_connections 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f8dc56e590 0x55f8dc9aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55f8dc5663b0 0x55f8dc93c0c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55f8dc58f270 0x55f8dc9b3c10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 --2- 172.21.15.112:0/2068142470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8dc54de40 0x55f8dc93bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 >> 172.21.15.112:0/2068142470 conn(0x55f8dc5833c0 msgr2=0x55f8dc93a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72cb7fe640 0 client.15108 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:37:55.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 shutdown_connections 2024-04-25T16:37:55.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 1 -- 172.21.15.112:0/2068142470 wait complete. 2024-04-25T16:37:55.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f9649740 20 client.15108 trim_cache size 0 max 16384 2024-04-25T16:37:55.180 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24560, 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 87.963002360999994, 2024-04-25T16:37:55.181 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-04-25T16:37:55.182 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 77, 2024-04-25T16:37:55.182 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T16:37:55.182 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 94.914478935000005 2024-04-25T16:37:55.182 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:37:55.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72d40afaf0 msgr2=0x7f72d40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f2796640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72d40afaf0 0x7f72d40b2000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f72e40ba240 tx=0x7f72e40972e0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 msgr2=0x7f72ec1a16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.157+0000 7f72f2796640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec1a16d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f72e40097b0 tx=0x7f72e4031840 comp rx=0 tx=0).stop 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 shutdown_connections 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72d40afaf0 0x7f72d40b2000 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72ec104d30 0x7f72ec19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f72ec109a30 0x7f72ec1a16d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 --2- 172.21.15.112:0/1884379098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ec103af0 0x7f72ec19a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 >> 172.21.15.112:0/1884379098 conn(0x7f72ec0fece0 msgr2=0x7f72ec105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 shutdown_connections 2024-04-25T16:37:55.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:55.158+0000 7f72f2796640 1 -- 172.21.15.112:0/1884379098 wait complete. 2024-04-25T16:37:55.185 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T16:37:55.185 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T16:37:55.195 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:37:55.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T16:37:55.431 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:55.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:55 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/806405236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:55.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:55 smithi112 ceph-mon[29122]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:55.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:55 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/806405236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:55.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:55 smithi133 ceph-mon[32691]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:56.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:55 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/806405236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:37:56.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:55 smithi116 ceph-mon[32712]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:37:57.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 -- 172.21.15.112:0/1855134643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850103680 msgr2=0x7f7850103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 --2- 172.21.15.112:0/1855134643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850103680 0x7f7850103a90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7840009990 tx=0x7f7840009ca0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 -- 172.21.15.112:0/1855134643 shutdown_connections 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 --2- 172.21.15.112:0/1855134643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7850104bc0 0x7f785010b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 --2- 172.21.15.112:0/1855134643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850103680 0x7f7850103a90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 --2- 172.21.15.112:0/1855134643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7850104060 0x7f7850104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 -- 172.21.15.112:0/1855134643 >> 172.21.15.112:0/1855134643 conn(0x7f78500fded0 msgr2=0x7f7850100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 -- 172.21.15.112:0/1855134643 shutdown_connections 2024-04-25T16:37:57.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.229+0000 7f78574dd700 1 -- 172.21.15.112:0/1855134643 wait complete. 2024-04-25T16:37:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 Processor -- start 2024-04-25T16:37:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 -- start start 2024-04-25T16:37:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7850103680 0x7f7850199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 0x7f7850199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:57.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7850104bc0 0x7f78501a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f785010ec50 con 0x7f7850104060 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f785010ead0 con 0x7f7850103680 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f78574dd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f785010edd0 con 0x7f7850104bc0 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 0x7f7850199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 0x7f7850199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38556/0 (socket says 172.21.15.112:38556) 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 -- 172.21.15.112:0/2898986301 learned_addr learned my addr 172.21.15.112:0/2898986301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7855a7a700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7850104bc0 0x7f78501a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7855279700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7850103680 0x7f7850199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 -- 172.21.15.112:0/2898986301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7850104bc0 msgr2=0x7f78501a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7850104bc0 0x7f78501a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 -- 172.21.15.112:0/2898986301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7850103680 msgr2=0x7f7850199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7850103680 0x7f7850199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.230+0000 7f7854a78700 1 -- 172.21.15.112:0/2898986301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78501a1360 con 0x7f7850104060 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.231+0000 7f7854a78700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 0x7f7850199d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b530 tx=0x7f784c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:57.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.231+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c015020 con 0x7f7850104060 2024-04-25T16:37:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.231+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501a1650 con 0x7f7850104060 2024-04-25T16:37:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.231+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f784c011ce0 con 0x7f7850104060 2024-04-25T16:37:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.231+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c010dd0 con 0x7f7850104060 2024-04-25T16:37:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.231+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78501a1c10 con 0x7f7850104060 2024-04-25T16:37:57.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.232+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7850051190 con 0x7f7850104060 2024-04-25T16:37:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.233+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f784c018020 con 0x7f7850104060 2024-04-25T16:37:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.234+0000 7f78467fc700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f783c0afb80 0x7f783c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:37:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.234+0000 7f7855279700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f783c0afb80 0x7f783c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:37:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.234+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f784c0de8e0 con 0x7f7850104060 2024-04-25T16:37:57.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.234+0000 7f7855279700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f783c0afb80 0x7f783c0b2090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f784000b5c0 tx=0x7f784001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:37:57.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.237+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f784c01f020 con 0x7f7850104060 2024-04-25T16:37:57.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.452+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f7850063ad0 con 0x7f783c0afb80 2024-04-25T16:37:57.452 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.454+0000 7f78467fc700 1 -- 172.21.15.112:0/2898986301 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f7850063ad0 con 0x7f783c0afb80 2024-04-25T16:37:57.452 INFO:teuthology.orchestra.run.smithi112.stdout:Backend: cephadm 2024-04-25T16:37:57.452 INFO:teuthology.orchestra.run.smithi112.stdout:Available: Yes 2024-04-25T16:37:57.452 INFO:teuthology.orchestra.run.smithi112.stdout:Paused: No 2024-04-25T16:37:57.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f783c0afb80 msgr2=0x7f783c0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:57.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f783c0afb80 0x7f783c0b2090 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f784000b5c0 tx=0x7f784001a040 comp rx=0 tx=0).stop 2024-04-25T16:37:57.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 msgr2=0x7f7850199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 0x7f7850199d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b530 tx=0x7f784c00b840 comp rx=0 tx=0).stop 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 shutdown_connections 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7850104bc0 0x7f78501a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f783c0afb80 0x7f783c0b2090 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7850103680 0x7f7850199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 --2- 172.21.15.112:0/2898986301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7850104060 0x7f7850199d90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 >> 172.21.15.112:0/2898986301 conn(0x7f78500fded0 msgr2=0x7f7850107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 shutdown_connections 2024-04-25T16:37:57.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:37:57.457+0000 7f78574dd700 1 -- 172.21.15.112:0/2898986301 wait complete. 2024-04-25T16:37:58.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T16:37:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:37:58.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:58 smithi133 ceph-mon[32691]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:37:58.486 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:58 smithi112 ceph-mon[29122]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:37:58.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:58 smithi116 ceph-mon[32712]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T16:37:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:37:59 smithi133 ceph-mon[32691]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:37:59.495 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:37:59 smithi112 ceph-mon[29122]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:37:59.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:37:59 smithi116 ceph-mon[32712]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:00.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 -- 172.21.15.112:0/1888023374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104be0 msgr2=0x7fce3410b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 --2- 172.21.15.112:0/1888023374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104be0 0x7fce3410b490 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fce30009990 tx=0x7fce30009ca0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 -- 172.21.15.112:0/1888023374 shutdown_connections 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 --2- 172.21.15.112:0/1888023374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104be0 0x7fce3410b490 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 --2- 172.21.15.112:0/1888023374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce34103110 0x7fce34103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 --2- 172.21.15.112:0/1888023374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce34104240 0x7fce34104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 -- 172.21.15.112:0/1888023374 >> 172.21.15.112:0/1888023374 conn(0x7fce340fe750 msgr2=0x7fce34100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 -- 172.21.15.112:0/1888023374 shutdown_connections 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.054+0000 7fce3b273700 1 -- 172.21.15.112:0/1888023374 wait complete. 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 Processor -- start 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 -- start start 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce34103110 0x7fce3419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:00.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 0x7fce3419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce34104be0 0x7fce341a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce3410fcf0 con 0x7fce34104240 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce3410fb70 con 0x7fce34104be0 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3b273700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce3410fe70 con 0x7fce34103110 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 0x7fce3419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 0x7fce3419acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38576/0 (socket says 172.21.15.112:38576) 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 -- 172.21.15.112:0/573314788 learned_addr learned my addr 172.21.15.112:0/573314788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 -- 172.21.15.112:0/573314788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce34103110 msgr2=0x7fce3419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce34103110 0x7fce3419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 -- 172.21.15.112:0/573314788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce34104be0 msgr2=0x7fce341a1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce34104be0 0x7fce341a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.055+0000 7fce3880e700 1 -- 172.21.15.112:0/573314788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce341a2240 con 0x7fce34104240 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.056+0000 7fce3880e700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 0x7fce3419acb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c9b0 tx=0x7fce2400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:00.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.056+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24007b00 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.056+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce24007ca0 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.056+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce341a2530 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.056+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce24018730 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.056+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce341a2ac0 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.057+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce34051190 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.058+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fce2400f450 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.059+0000 7fce2a7fc700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fce200afc00 0x7fce200b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.059+0000 7fce3900f700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fce200afc00 0x7fce200b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.059+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fce240def50 con 0x7fce34104240 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.059+0000 7fce3900f700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fce200afc00 0x7fce200b2110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce34104140 tx=0x7fce2c0093a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:00.086 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.062+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce24014030 con 0x7fce34104240 2024-04-25T16:38:00.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.271+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fce34063ad0 con 0x7fce200afc00 2024-04-25T16:38:00.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.284+0000 7fce2a7fc700 1 -- 172.21.15.112:0/573314788 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fce34063ad0 con 0x7fce200afc00 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.a smithi112 running (103s) 15s ago 102s 83.8M - 19.0.0-3291-g994e2f20 d53c96779a29 8f1f0b22835a 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.b smithi116 running (100s) 34s ago 99s 92.9M - 19.0.0-3291-g994e2f20 d53c96779a29 755e27b715b3 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.c smithi133 running (114s) 34s ago 114s 85.8M - 19.0.0-3291-g994e2f20 d53c96779a29 c5cc1be113a3 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.d smithi112 running (2m) 15s ago 2m 81.6M - 19.0.0-3291-g994e2f20 d53c96779a29 b9ba950d77be 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.e smithi116 running (2m) 34s ago 119s 87.1M - 19.0.0-3291-g994e2f20 d53c96779a29 a56bba8a68dd 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.f smithi133 running (117s) 34s ago 117s 84.5M - 19.0.0-3291-g994e2f20 d53c96779a29 b1e19a565213 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.g smithi112 running (106s) 15s ago 105s 81.2M - 19.0.0-3291-g994e2f20 d53c96779a29 0361e0ebd79f 2024-04-25T16:38:00.283 INFO:teuthology.orchestra.run.smithi112.stdout:mds.h smithi116 running (111s) 34s ago 111s 88.1M - 19.0.0-3291-g994e2f20 d53c96779a29 bd9c689a7e37 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mds.i smithi133 running (108s) 34s ago 108s 87.5M - 19.0.0-3291-g994e2f20 d53c96779a29 a2a3658ee3f0 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mds.j smithi112 running (2m) 15s ago 2m 80.7M - 19.0.0-3291-g994e2f20 d53c96779a29 c264df212309 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mds.k smithi116 running (2m) 34s ago 2m 80.8M - 19.0.0-3291-g994e2f20 d53c96779a29 627db4aa9bea 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mds.l smithi133 running (2m) 34s ago 2m 85.9M - 19.0.0-3291-g994e2f20 d53c96779a29 13692d82e059 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.x smithi112 *:9283,8765 running (9m) 15s ago 9m 585M - 19.0.0-3291-g994e2f20 d53c96779a29 d8c4e38aecbc 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.y smithi116 *:8443,8765 running (7m) 34s ago 7m 522M - 19.0.0-3291-g994e2f20 d53c96779a29 77c04953f41c 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mgr.z smithi133 *:8443,8765 running (7m) 34s ago 7m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 b0b3a1821e60 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mon.a smithi112 running (9m) 15s ago 9m 116M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 60a3b4a63620 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mon.b smithi116 running (7m) 34s ago 7m 109M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 dd884f0c9137 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:mon.c smithi133 running (8m) 34s ago 8m 107M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 144bc0b8c319 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 smithi112 running (7m) 15s ago 7m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 13a0259ea1b5 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 smithi116 running (6m) 34s ago 6m 113M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f850961f5128 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 smithi133 running (6m) 34s ago 6m 115M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 f833ac8eb860 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 smithi112 running (6m) 15s ago 6m 112M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 9955c603034d 2024-04-25T16:38:00.284 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 smithi116 running (5m) 34s ago 5m 114M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 9d9111c54140 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 smithi133 running (5m) 34s ago 5m 120M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 0aa3eebda048 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 smithi112 running (4m) 15s ago 4m 121M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 46b50f21cd5d 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 smithi116 running (4m) 34s ago 4m 111M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 727f721aa43d 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 smithi133 running (3m) 34s ago 3m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 88e4787846cb 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 smithi112 running (3m) 15s ago 3m 118M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 bd387d1427a0 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 smithi116 running (2m) 34s ago 2m 110M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 2e88cce8072c 2024-04-25T16:38:00.285 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 smithi133 running (2m) 34s ago 2m 109M 4272M 19.0.0-3291-g994e2f20 d53c96779a29 75544e02ef48 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fce200afc00 msgr2=0x7fce200b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fce200afc00 0x7fce200b2110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce34104140 tx=0x7fce2c0093a0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 msgr2=0x7fce3419acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 0x7fce3419acb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fce2400c9b0 tx=0x7fce2400cd70 comp rx=0 tx=0).stop 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 shutdown_connections 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fce200afc00 0x7fce200b2110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce34104be0 0x7fce341a1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce34103110 0x7fce3419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 --2- 172.21.15.112:0/573314788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce34104240 0x7fce3419acb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:00.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.286+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 >> 172.21.15.112:0/573314788 conn(0x7fce340fe750 msgr2=0x7fce34107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.287+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 shutdown_connections 2024-04-25T16:38:00.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:00.287+0000 7fce3b273700 1 -- 172.21.15.112:0/573314788 wait complete. 2024-04-25T16:38:00.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:00 smithi112 ceph-mon[29122]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:38:00.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:00 smithi133 ceph-mon[32691]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:38:00.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:00 smithi116 ceph-mon[32712]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T16:38:01.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T16:38:01.374 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:38:02.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:02 smithi112 ceph-mon[29122]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:02.432 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:02 smithi112 ceph-mon[29122]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:38:02.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:02 smithi133 ceph-mon[32691]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:02.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:02 smithi133 ceph-mon[32691]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:02 smithi116 ceph-mon[32712]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:02.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:02 smithi116 ceph-mon[32712]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:38:03.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 -- 172.21.15.112:0/2714803728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254105cf0 msgr2=0x7f725410c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:03.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 --2- 172.21.15.112:0/2714803728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254105cf0 0x7f725410c5a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f724c00b0a0 tx=0x7f724c00b3b0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 -- 172.21.15.112:0/2714803728 shutdown_connections 2024-04-25T16:38:03.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 --2- 172.21.15.112:0/2714803728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254105cf0 0x7f725410c5a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 --2- 172.21.15.112:0/2714803728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254104220 0x7f7254104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 --2- 172.21.15.112:0/2714803728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7254105350 0x7f7254105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 -- 172.21.15.112:0/2714803728 >> 172.21.15.112:0/2714803728 conn(0x7f72540ff860 msgr2=0x7f7254101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 -- 172.21.15.112:0/2714803728 shutdown_connections 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.024+0000 7f725ad49700 1 -- 172.21.15.112:0/2714803728 wait complete. 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 Processor -- start 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 -- start start 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 0x7f725419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254105350 0x7f725419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7254105cf0 0x7f72541a1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f725410fc80 con 0x7f7254104220 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f725410fb00 con 0x7f7254105350 2024-04-25T16:38:03.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f725ad49700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f725410fe00 con 0x7f7254105cf0 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f7258ae5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 0x7f725419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f7258ae5700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 0x7f725419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38592/0 (socket says 172.21.15.112:38592) 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.025+0000 7f7258ae5700 1 -- 172.21.15.112:0/1222553988 learned_addr learned my addr 172.21.15.112:0/1222553988 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f72592e6700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7254105cf0 0x7f72541a1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7253fff700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254105350 0x7f725419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7258ae5700 1 -- 172.21.15.112:0/1222553988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7254105cf0 msgr2=0x7f72541a1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7258ae5700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7254105cf0 0x7f72541a1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7258ae5700 1 -- 172.21.15.112:0/1222553988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254105350 msgr2=0x7f725419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7258ae5700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254105350 0x7f725419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7258ae5700 1 -- 172.21.15.112:0/1222553988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72541a22c0 con 0x7f7254104220 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7253fff700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254105350 0x7f725419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7258ae5700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 0x7f725419a760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f724800d950 tx=0x7f724800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:03.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248017070 con 0x7f7254104220 2024-04-25T16:38:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7248009730 con 0x7f7254104220 2024-04-25T16:38:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248003a50 con 0x7f7254104220 2024-04-25T16:38:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.026+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72541a25b0 con 0x7f7254104220 2024-04-25T16:38:03.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.027+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72541a2af0 con 0x7f7254104220 2024-04-25T16:38:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.028+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725402e510 con 0x7f7254104220 2024-04-25T16:38:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.029+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f7248003bf0 con 0x7f7254104220 2024-04-25T16:38:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.029+0000 7f7251ffb700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72440afc00 0x7f72440b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.029+0000 7f7253fff700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72440afc00 0x7f72440b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.029+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f72480de6b0 con 0x7f7254104220 2024-04-25T16:38:03.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.030+0000 7f7253fff700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72440afc00 0x7f72440b2110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7240005950 tx=0x7f72400058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:03.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.034+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7248020030 con 0x7f7254104220 2024-04-25T16:38:03.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.243+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7254063ad0 con 0x7f72440afc00 2024-04-25T16:38:03.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.249+0000 7f7251ffb700 1 -- 172.21.15.112:0/1222553988 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7254063ad0 con 0x7f72440afc00 2024-04-25T16:38:03.247 INFO:teuthology.orchestra.run.smithi112.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T16:38:03.248 INFO:teuthology.orchestra.run.smithi112.stdout:mds.all 12/12 37s ago 2m smithi112=a;smithi112=d;smithi112=g;smithi112=j;smithi116=b;smithi116=e;smithi116=h;smithi116=k;smithi133=c;smithi133=f;smithi133=i;smithi133=l;count:12 2024-04-25T16:38:03.248 INFO:teuthology.orchestra.run.smithi112.stdout:mgr 3/3 37s ago 7m smithi112=x;smithi116=y;smithi133=z;count:3 2024-04-25T16:38:03.248 INFO:teuthology.orchestra.run.smithi112.stdout:mon 3/3 37s ago 8m smithi112:172.21.15.112=a;smithi116:172.21.15.116=b;smithi133:172.21.15.133=c;count:3 2024-04-25T16:38:03.248 INFO:teuthology.orchestra.run.smithi112.stdout:osd 12 37s ago - 2024-04-25T16:38:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72440afc00 msgr2=0x7f72440b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72440afc00 0x7f72440b2110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7240005950 tx=0x7f72400058e0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 msgr2=0x7f725419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 0x7f725419a760 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f724800d950 tx=0x7f724800dc60 comp rx=0 tx=0).stop 2024-04-25T16:38:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 shutdown_connections 2024-04-25T16:38:03.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7254105cf0 0x7f72541a1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7254104220 0x7f725419a760 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f72440afc00 0x7f72440b2110 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 --2- 172.21.15.112:0/1222553988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7254105350 0x7f725419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:03.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.251+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 >> 172.21.15.112:0/1222553988 conn(0x7f72540ff860 msgr2=0x7f7254108580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:03.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.252+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 shutdown_connections 2024-04-25T16:38:03.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:03.252+0000 7f725ad49700 1 -- 172.21.15.112:0/1222553988 wait complete. 2024-04-25T16:38:03.945 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T16:38:04.187 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:38:04.431 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:04 smithi112 ceph-mon[29122]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:38:04.432 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:04 smithi112 ceph-mon[29122]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:04.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:04 smithi133 ceph-mon[32691]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:38:04.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:04 smithi133 ceph-mon[32691]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:04.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:04 smithi116 ceph-mon[32712]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:38:04.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:04 smithi116 ceph-mon[32712]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 -- 172.21.15.112:0/3769298219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 msgr2=0x7f0880105bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 --2- 172.21.15.112:0/3769298219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f0880105bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f087c009990 tx=0x7f087c009ca0 comp rx=0 tx=0).stop 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 -- 172.21.15.112:0/3769298219 shutdown_connections 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 --2- 172.21.15.112:0/3769298219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0880102b40 0x7f088010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 --2- 172.21.15.112:0/3769298219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f0880105bb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 --2- 172.21.15.112:0/3769298219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08801021f0 0x7f0880102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 -- 172.21.15.112:0/3769298219 >> 172.21.15.112:0/3769298219 conn(0x7f08800fdec0 msgr2=0x7f0880100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:05.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 -- 172.21.15.112:0/3769298219 shutdown_connections 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.851+0000 7f0887d92700 1 -- 172.21.15.112:0/3769298219 wait complete. 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 Processor -- start 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 -- start start 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08801021f0 0x7f088019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0880102b40 0x7f088019ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f08801a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f088010fd40 con 0x7f08801057a0 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f088010fbc0 con 0x7f08801021f0 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0887d92700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f088010fec0 con 0x7f0880102b40 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f08801a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f08801a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53620/0 (socket says 172.21.15.112:53620) 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 -- 172.21.15.112:0/2420159868 learned_addr learned my addr 172.21.15.112:0/2420159868 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:05.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0885b2e700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08801021f0 0x7f088019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088532d700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0880102b40 0x7f088019ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 -- 172.21.15.112:0/2420159868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0880102b40 msgr2=0x7f088019ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0880102b40 0x7f088019ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 -- 172.21.15.112:0/2420159868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08801021f0 msgr2=0x7f088019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08801021f0 0x7f088019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088632f700 1 -- 172.21.15.112:0/2420159868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08801a24b0 con 0x7f08801057a0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f088532d700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0880102b40 0x7f088019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.852+0000 7f0885b2e700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08801021f0 0x7f088019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.853+0000 7f088632f700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f08801a1e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f087800b970 tx=0x7f087800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.853+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878019070 con 0x7f08801057a0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.853+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0878014440 con 0x7f08801057a0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.853+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08801a27a0 con 0x7f08801057a0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.853+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087800ce60 con 0x7f08801057a0 2024-04-25T16:38:05.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.853+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a2d60 con 0x7f08801057a0 2024-04-25T16:38:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.854+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0880051190 con 0x7f08801057a0 2024-04-25T16:38:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.855+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f087801c070 con 0x7f08801057a0 2024-04-25T16:38:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.856+0000 7f0876ffd700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f086c0afc00 0x7f086c0b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.856+0000 7f0885b2e700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f086c0afc00 0x7f086c0b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.856+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f08780de500 con 0x7f08801057a0 2024-04-25T16:38:05.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.856+0000 7f0885b2e700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f086c0afc00 0x7f086c0b2110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f087c00b5c0 tx=0x7f087c005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:05.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:05.859+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08780a70c0 con 0x7f08801057a0 2024-04-25T16:38:06.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.063+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0880063ad0 con 0x7f086c0afc00 2024-04-25T16:38:06.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.064+0000 7f0876ffd700 1 -- 172.21.15.112:0/2420159868 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0880063ad0 con 0x7f086c0afc00 2024-04-25T16:38:06.063 INFO:teuthology.orchestra.run.smithi112.stdout:HOST ADDR LABELS STATUS 2024-04-25T16:38:06.063 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 172.21.15.112 2024-04-25T16:38:06.063 INFO:teuthology.orchestra.run.smithi112.stdout:smithi116 172.21.15.116 2024-04-25T16:38:06.063 INFO:teuthology.orchestra.run.smithi112.stdout:smithi133 172.21.15.133 2024-04-25T16:38:06.063 INFO:teuthology.orchestra.run.smithi112.stdout:3 hosts in cluster 2024-04-25T16:38:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.066+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f086c0afc00 msgr2=0x7f086c0b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.066+0000 7f0887d92700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f086c0afc00 0x7f086c0b2110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f087c00b5c0 tx=0x7f087c005a90 comp rx=0 tx=0).stop 2024-04-25T16:38:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.066+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 msgr2=0x7f08801a1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.066+0000 7f0887d92700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f08801a1e30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f087800b970 tx=0x7f087800bd30 comp rx=0 tx=0).stop 2024-04-25T16:38:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 shutdown_connections 2024-04-25T16:38:06.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08801057a0 0x7f08801a1e30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f086c0afc00 0x7f086c0b2110 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08801021f0 0x7f088019a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 --2- 172.21.15.112:0/2420159868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0880102b40 0x7f088019ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 >> 172.21.15.112:0/2420159868 conn(0x7f08800fdec0 msgr2=0x7f08801087b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 shutdown_connections 2024-04-25T16:38:06.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:06.067+0000 7f0887d92700 1 -- 172.21.15.112:0/2420159868 wait complete. 2024-04-25T16:38:06.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:06 smithi133 ceph-mon[32691]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:38:06.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:06 smithi116 ceph-mon[32712]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:38:06.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:06 smithi112 ceph-mon[29122]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:38:06.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T16:38:07.054 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:38:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:07 smithi133 ceph-mon[32691]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:07.454 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:07 smithi112 ceph-mon[29122]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:07 smithi116 ceph-mon[32712]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:08.390 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:08 smithi112 ceph-mon[29122]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:38:08.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:08 smithi133 ceph-mon[32691]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:38:08.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:08 smithi116 ceph-mon[32712]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:38:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.636+0000 7f0c1122e700 1 -- 172.21.15.112:0/2158136126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105ce0 msgr2=0x7f0c0c10c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:08.635 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.636+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2158136126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105ce0 0x7f0c0c10c590 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b0a0 tx=0x7f0c0400b3b0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 -- 172.21.15.112:0/2158136126 shutdown_connections 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2158136126 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105ce0 0x7f0c0c10c590 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2158136126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0c104210 0x7f0c0c104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2158136126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0c105340 0x7f0c0c105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 -- 172.21.15.112:0/2158136126 >> 172.21.15.112:0/2158136126 conn(0x7f0c0c0ff870 msgr2=0x7f0c0c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 -- 172.21.15.112:0/2158136126 shutdown_connections 2024-04-25T16:38:08.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 -- 172.21.15.112:0/2158136126 wait complete. 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.637+0000 7f0c1122e700 1 Processor -- start 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 -- start start 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0c104210 0x7f0c0c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 0x7f0c0c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0c105ce0 0x7f0c0c1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10fc90 con 0x7f0c0c105340 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10fb10 con 0x7f0c0c105ce0 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c1122e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c10fe10 con 0x7f0c0c104210 2024-04-25T16:38:08.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 0x7f0c0c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 0x7f0c0c19ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53636/0 (socket says 172.21.15.112:53636) 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 -- 172.21.15.112:0/2825573487 learned_addr learned my addr 172.21.15.112:0/2825573487 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0ad9d700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0c104210 0x7f0c0c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0b59e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0c105ce0 0x7f0c0c1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 -- 172.21.15.112:0/2825573487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0c104210 msgr2=0x7f0c0c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0c104210 0x7f0c0c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 -- 172.21.15.112:0/2825573487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0c105ce0 msgr2=0x7f0c0c1a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0c105ce0 0x7f0c0c1a1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.638+0000 7f0c0a59c700 1 -- 172.21.15.112:0/2825573487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c1a2280 con 0x7f0c0c105340 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.639+0000 7f0c0a59c700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 0x7f0c0c19ac60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000c970 tx=0x7f0c0000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.639+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c000078d0 con 0x7f0c0c105340 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.639+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c00018480 con 0x7f0c0c105340 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.639+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c1a2570 con 0x7f0c0c105340 2024-04-25T16:38:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.639+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0000f6b0 con 0x7f0c0c105340 2024-04-25T16:38:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.639+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c1a2a60 con 0x7f0c0c105340 2024-04-25T16:38:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.640+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c051190 con 0x7f0c0c105340 2024-04-25T16:38:08.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.642+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f0c00018930 con 0x7f0c0c105340 2024-04-25T16:38:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.642+0000 7f0bf3fff700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f0bf40afb80 0x7f0bf40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.643+0000 7f0c0ad9d700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f0bf40afb80 0x7f0bf40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.643+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f0c000df010 con 0x7f0c0c105340 2024-04-25T16:38:08.641 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.643+0000 7f0c0ad9d700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f0bf40afb80 0x7f0bf40b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c105240 tx=0x7f0bfc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.646+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c000a7cf0 con 0x7f0c0c105340 2024-04-25T16:38:08.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.853+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0c0c063ad0 con 0x7f0bf40afb80 2024-04-25T16:38:08.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.855+0000 7f0bf3fff700 1 -- 172.21.15.112:0/2825573487 <== mgr.14152 v2:172.21.15.112:6800/3947590362 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0c0c063ad0 con 0x7f0bf40afb80 2024-04-25T16:38:08.853 INFO:teuthology.orchestra.run.smithi112.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T16:38:08.853 INFO:teuthology.orchestra.run.smithi112.stdout:smithi112 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NV400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T16:38:08.853 INFO:teuthology.orchestra.run.smithi112.stdout:smithi116 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T16:38:08.853 INFO:teuthology.orchestra.run.smithi112.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f0bf40afb80 msgr2=0x7f0bf40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f0bf40afb80 0x7f0bf40b2090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c105240 tx=0x7f0bfc00a380 comp rx=0 tx=0).stop 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 msgr2=0x7f0c0c19ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 0x7f0c0c19ac60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0c0000c970 tx=0x7f0c0000cc80 comp rx=0 tx=0).stop 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 shutdown_connections 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f0bf40afb80 0x7f0bf40b2090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c0c105ce0 0x7f0c0c1a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0c104210 0x7f0c0c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 --2- 172.21.15.112:0/2825573487 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0c0c105340 0x7f0c0c19ac60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:08.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 >> 172.21.15.112:0/2825573487 conn(0x7f0c0c0ff870 msgr2=0x7f0c0c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:08.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 shutdown_connections 2024-04-25T16:38:08.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:08.857+0000 7f0c1122e700 1 -- 172.21.15.112:0/2825573487 wait complete. 2024-04-25T16:38:09.639 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T16:38:09.648 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:38:09.648 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T16:38:09.888 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:38:10.314 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:10 smithi112 ceph-mon[29122]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:10.315 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:10 smithi112 ceph-mon[29122]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T16:38:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:10 smithi133 ceph-mon[32691]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:10 smithi133 ceph-mon[32691]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T16:38:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:10 smithi116 ceph-mon[32712]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:10.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:10 smithi116 ceph-mon[32712]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T16:38:11.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 -- 172.21.15.112:0/3236823999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105cf0 msgr2=0x7f932810c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 --2- 172.21.15.112:0/3236823999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105cf0 0x7f932810c5a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f932000b0a0 tx=0x7f932000b3b0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 -- 172.21.15.112:0/3236823999 shutdown_connections 2024-04-25T16:38:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 --2- 172.21.15.112:0/3236823999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105cf0 0x7f932810c5a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 --2- 172.21.15.112:0/3236823999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328104220 0x7f9328104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 --2- 172.21.15.112:0/3236823999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328105350 0x7f9328105760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 -- 172.21.15.112:0/3236823999 >> 172.21.15.112:0/3236823999 conn(0x7f93280ff860 msgr2=0x7f9328101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 -- 172.21.15.112:0/3236823999 shutdown_connections 2024-04-25T16:38:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.394+0000 7f932e4cb700 1 -- 172.21.15.112:0/3236823999 wait complete. 2024-04-25T16:38:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 Processor -- start 2024-04-25T16:38:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 -- start start 2024-04-25T16:38:11.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328104220 0x7f932819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 0x7f932819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328105cf0 0x7f93281a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f932810fd00 con 0x7f9328105350 2024-04-25T16:38:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f932810fb80 con 0x7f9328105cf0 2024-04-25T16:38:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.395+0000 7f932e4cb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f932810fe80 con 0x7f9328104220 2024-04-25T16:38:11.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 0x7f932819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 0x7f932819cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53654/0 (socket says 172.21.15.112:53654) 2024-04-25T16:38:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 -- 172.21.15.112:0/1209732848 learned_addr learned my addr 172.21.15.112:0/1209732848 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f9327fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328104220 0x7f932819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f932ca68700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328105cf0 0x7f93281a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 -- 172.21.15.112:0/1209732848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328104220 msgr2=0x7f932819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:11.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328104220 0x7f932819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 -- 172.21.15.112:0/1209732848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328105cf0 msgr2=0x7f93281a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328105cf0 0x7f93281a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 -- 172.21.15.112:0/1209732848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93281a44d0 con 0x7f9328105350 2024-04-25T16:38:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f9327fff700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328104220 0x7f932819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:38:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f932ca68700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328105cf0 0x7f93281a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:38:11.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93277fe700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 0x7f932819cf40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f931800b530 tx=0x7f931800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318015020 con 0x7f9328105350 2024-04-25T16:38:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93281a47c0 con 0x7f9328105350 2024-04-25T16:38:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9318011ce0 con 0x7f9328105350 2024-04-25T16:38:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.396+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9318010dd0 con 0x7f9328105350 2024-04-25T16:38:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.397+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93281a4d80 con 0x7f9328105350 2024-04-25T16:38:11.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.398+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9328051190 con 0x7f9328105350 2024-04-25T16:38:11.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.399+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f9318018020 con 0x7f9328105350 2024-04-25T16:38:11.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.400+0000 7f93257fa700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f93100afb80 0x7f93100b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:11.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.400+0000 7f9327fff700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f93100afb80 0x7f93100b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:11.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.400+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f93180de8e0 con 0x7f9328105350 2024-04-25T16:38:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.400+0000 7f9327fff700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f93100afb80 0x7f93100b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9328105250 tx=0x7f931c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:11.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.403+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f931801f020 con 0x7f9328105350 2024-04-25T16:38:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.644+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f93281a51a0 con 0x7f9328105350 2024-04-25T16:38:11.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.645+0000 7f93257fa700 1 -- 172.21.15.112:0/1209732848 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3054 (secure 0 0 0) 0x7f93180a74a0 con 0x7f9328105350 2024-04-25T16:38:11.643 INFO:teuthology.orchestra.run.smithi112.stdout:e21 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:legacy client fscid: 1 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:Filesystem 'cephfs' (1) 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:fs_name cephfs 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 21 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-25T16:36:26.563199+0000 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-04-25T16:36:31.235238+0000 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:tableserver 0 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:root 0 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:session_timeout 300 2024-04-25T16:38:11.644 INFO:teuthology.orchestra.run.smithi112.stdout:session_autoclose 300 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:max_file_size 1099511627776 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:max_xattr_size 65536 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:required_client_features {} 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure 0 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:last_failure_osd_epoch 0 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:max_mds 3 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:in 0,1,2 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:up {0=24560,1=24524,2=24514} 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:failed 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:damaged 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:stopped 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:data_pools [3] 2024-04-25T16:38:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:metadata_pool 2 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:inline_data disabled 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:balancer 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:bal_rank_mask -1 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:standby_count_wanted 0 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:qdb_cluster leader: 24560 members: 24514,24524,24560 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b{0:24560} state up:active seq 3 addr [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.e{0:24488} state up:standby-replay seq 1 addr [v2:172.21.15.116:6834/4202555064,v1:172.21.15.116:6835/4202555064] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.h{1:24524} state up:active seq 7 addr [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c{1:24484} state up:standby-replay seq 1 addr [v2:172.21.15.133:6836/3840802165,v1:172.21.15.133:6837/3840802165] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.i{2:24514} state up:active seq 6 addr [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.f{2:24472} state up:standby-replay seq 1 addr [v2:172.21.15.133:6834/2304762610,v1:172.21.15.133:6835/2304762610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:11.646 INFO:teuthology.orchestra.run.smithi112.stdout:Standby daemons: 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.j{-1:14547} state up:standby seq 1 addr [v2:172.21.15.112:6834/759452882,v1:172.21.15.112:6835/759452882] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.112:6836/1910235790,v1:172.21.15.112:6837/1910235790] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.g{-1:14628} state up:standby seq 1 addr [v2:172.21.15.112:6838/72433575,v1:172.21.15.112:6839/72433575] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a{-1:14640} state up:standby seq 1 addr [v2:172.21.15.112:6840/4208559487,v1:172.21.15.112:6841/4208559487] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.133:6832/2075237111,v1:172.21.15.133:6833/2075237111] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.647 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.116:6832/2123343031,v1:172.21.15.116:6833/2123343031] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:38:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f93100afb80 msgr2=0x7f93100b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f93100afb80 0x7f93100b2090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9328105250 tx=0x7f931c00a3b0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 msgr2=0x7f932819cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 0x7f932819cf40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f931800b530 tx=0x7f931800b840 comp rx=0 tx=0).stop 2024-04-25T16:38:11.651 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 shutdown_connections 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f93100afb80 0x7f93100b2090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9328105cf0 0x7f93281a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328104220 0x7f932819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 --2- 172.21.15.112:0/1209732848 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9328105350 0x7f932819cf40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 >> 172.21.15.112:0/1209732848 conn(0x7f93280ff860 msgr2=0x7f93280758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 shutdown_connections 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:11.648+0000 7f932e4cb700 1 -- 172.21.15.112:0/1209732848 wait complete. 2024-04-25T16:38:11.652 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:38:12.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:12 smithi112 ceph-mon[29122]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:38:12.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:12 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1209732848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T16:38:12.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a7905982-0320-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T16:38:12.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:12 smithi133 ceph-mon[32691]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:38:12.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:12 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1209732848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T16:38:12.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:12 smithi116 ceph-mon[32712]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T16:38:12.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:12 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1209732848' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T16:38:12.649 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 -- 172.21.15.112:0/1349083516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 msgr2=0x7f7fa010b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/1349083516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa010b650 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f980099a0 tx=0x7f7f98009cb0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 -- 172.21.15.112:0/1349083516 shutdown_connections 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/1349083516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa010b650 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/1349083516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0103680 0x7f7fa0103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/1349083516 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa0104060 0x7f7fa0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 -- 172.21.15.112:0/1349083516 >> 172.21.15.112:0/1349083516 conn(0x7f7fa00fded0 msgr2=0x7f7fa0100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 -- 172.21.15.112:0/1349083516 shutdown_connections 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.303+0000 7f7fa6f54700 1 -- 172.21.15.112:0/1349083516 wait complete. 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 Processor -- start 2024-04-25T16:38:14.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 -- start start 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa0103680 0x7f7fa01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0104060 0x7f7fa0199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa010ec00 con 0x7f7fa0104bc0 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa010ea80 con 0x7f7fa0103680 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa6f54700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa010ed80 con 0x7f7fa0104060 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa01a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53668/0 (socket says 172.21.15.112:53668) 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 -- 172.21.15.112:0/2563622627 learned_addr learned my addr 172.21.15.112:0/2563622627 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa4cf0700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa0103680 0x7f7fa01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7f97fff700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0104060 0x7f7fa0199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 -- 172.21.15.112:0/2563622627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0104060 msgr2=0x7f7fa0199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0104060 0x7f7fa0199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 -- 172.21.15.112:0/2563622627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa0103680 msgr2=0x7f7fa01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.304+0000 7f7fa54f1700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa0103680 0x7f7fa01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7fa54f1700 1 -- 172.21.15.112:0/2563622627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a1210 con 0x7f7fa0104bc0 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7f97fff700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0104060 0x7f7fa0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7fa54f1700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa01a0b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7f98005660 tx=0x7f7f98005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98026070 con 0x7f7fa0104bc0 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f98003d90 con 0x7f7fa0104bc0 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a14a0 con 0x7f7fa0104bc0 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f98013070 con 0x7f7fa0104bc0 2024-04-25T16:38:14.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.305+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a18f0 con 0x7f7fa0104bc0 2024-04-25T16:38:14.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.306+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0051190 con 0x7f7fa0104bc0 2024-04-25T16:38:14.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.308+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f7f98029070 con 0x7f7fa0104bc0 2024-04-25T16:38:14.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.308+0000 7f7f95ffb700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7f880afb80 0x7f7f880b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:14.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.308+0000 7f7fa4cf0700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7f880afb80 0x7f7f880b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:14.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.308+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f7f980dfef0 con 0x7f7fa0104bc0 2024-04-25T16:38:14.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.309+0000 7f7fa4cf0700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7f880afb80 0x7f7f880b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7fa006bf30 tx=0x7f7f9c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:14.310 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.312+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f98030070 con 0x7f7fa0104bc0 2024-04-25T16:38:14.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:14 smithi133 ceph-mon[32691]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:38:14.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:14 smithi116 ceph-mon[32712]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:38:14.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.511+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7fa002e510 con 0x7f7fa0104bc0 2024-04-25T16:38:14.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:14 smithi112 ceph-mon[29122]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T16:38:14.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.512+0000 7f7f95ffb700 1 -- 172.21.15.112:0/2563622627 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5365 (secure 0 0 0) 0x7f7f980a8b40 con 0x7f7fa0104bc0 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 77 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:fsid a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-25T16:28:12.063898+0000 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:modified 2024-04-25T16:37:16.199786+0000 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:crush_version 26 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:full_ratio 0.95 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:backfillfull_ratio 0.9 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:nearfull_ratio 0.85 2024-04-25T16:38:14.511 INFO:teuthology.orchestra.run.smithi112.stdout:require_min_compat_client luminous 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:min_compat_client luminous 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:require_osd_release squid 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:stretch_mode_enabled false 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 74 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T16:38:14.512 INFO:teuthology.orchestra.run.smithi112.stdout:max_osd 12 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.0 up in weight 1 up_from 8 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6802/2222102236,v1:172.21.15.112:6803/2222102236] [v2:172.21.15.112:6804/2222102236,v1:172.21.15.112:6805/2222102236] exists,up 2011d44b-d406-413a-aa94-1a3a50fcde37 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6800/2666280069,v1:172.21.15.116:6801/2666280069] [v2:172.21.15.116:6802/2666280069,v1:172.21.15.116:6803/2666280069] exists,up cb335e86-8675-4ab0-826c-9cf84a05a605 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.2 up in weight 1 up_from 18 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6800/1571915976,v1:172.21.15.133:6801/1571915976] [v2:172.21.15.133:6802/1571915976,v1:172.21.15.133:6803/1571915976] exists,up a075e9d7-1972-4815-8471-6a49816ec39f 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.3 up in weight 1 up_from 25 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6810/1393578642,v1:172.21.15.112:6811/1393578642] [v2:172.21.15.112:6812/1393578642,v1:172.21.15.112:6813/1393578642] exists,up 1d45175e-d218-4c74-bcec-a037fcb395f2 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.4 up in weight 1 up_from 31 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6808/2551746095,v1:172.21.15.116:6809/2551746095] [v2:172.21.15.116:6810/2551746095,v1:172.21.15.116:6811/2551746095] exists,up a0f9a634-8f5a-4dbe-a8e9-8f74872cb417 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.5 up in weight 1 up_from 36 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6808/3110277631,v1:172.21.15.133:6809/3110277631] [v2:172.21.15.133:6810/3110277631,v1:172.21.15.133:6811/3110277631] exists,up 3a17fcaf-3f2f-4fef-b666-a6c9927dc38a 2024-04-25T16:38:14.513 INFO:teuthology.orchestra.run.smithi112.stdout:osd.6 up in weight 1 up_from 42 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6818/4240993708,v1:172.21.15.112:6819/4240993708] [v2:172.21.15.112:6820/4240993708,v1:172.21.15.112:6821/4240993708] exists,up bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:osd.7 up in weight 1 up_from 47 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6816/590137148,v1:172.21.15.116:6817/590137148] [v2:172.21.15.116:6818/590137148,v1:172.21.15.116:6819/590137148] exists,up 79c1b323-2808-4b81-9164-36836a690a83 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:osd.8 up in weight 1 up_from 53 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6816/1430124311,v1:172.21.15.133:6817/1430124311] [v2:172.21.15.133:6818/1430124311,v1:172.21.15.133:6819/1430124311] exists,up c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:osd.9 up in weight 1 up_from 59 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.112:6826/2630590642,v1:172.21.15.112:6827/2630590642] [v2:172.21.15.112:6828/2630590642,v1:172.21.15.112:6829/2630590642] exists,up a29c6af5-9461-4eef-b445-bcb8a7bfed1e 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:osd.10 up in weight 1 up_from 64 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.116:6824/3599339690,v1:172.21.15.116:6825/3599339690] [v2:172.21.15.116:6826/3599339690,v1:172.21.15.116:6827/3599339690] exists,up b0d3168a-8529-49d8-bd4c-1c1bd7336f60 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:osd.11 up in weight 1 up_from 70 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6824/2381034021,v1:172.21.15.133:6825/2381034021] [v2:172.21.15.133:6826/2381034021,v1:172.21.15.133:6827/2381034021] exists,up f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 2.b [2,4] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.1 [4,9] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.6 [4,6] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.17 [4,6] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.1a [5,7] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.1b [3,8] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.24 [4,9] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.2d [3,9] 2024-04-25T16:38:14.514 INFO:teuthology.orchestra.run.smithi112.stdout:pg_upmap_items 3.37 [5,8] 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3651646348 expires 2024-04-26T16:28:48.092289+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/108124461 expires 2024-04-26T16:29:14.986280+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1519283708 expires 2024-04-26T16:28:48.092289+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/944410496 expires 2024-04-26T16:28:48.092289+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/1130097819 expires 2024-04-26T16:28:48.092289+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/3285899721 expires 2024-04-26T16:29:14.986280+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6800/108124461 expires 2024-04-26T16:29:14.986280+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:6801/1130097819 expires 2024-04-26T16:28:48.092289+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/404948490 expires 2024-04-26T16:29:14.986280+0000 2024-04-25T16:38:14.515 INFO:teuthology.orchestra.run.smithi112.stdout:blocklist 172.21.15.112:0/1942519556 expires 2024-04-26T16:29:14.986280+0000 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7f880afb80 msgr2=0x7f7f880b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7f880afb80 0x7f7f880b2090 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7fa006bf30 tx=0x7f7f9c00d040 comp rx=0 tx=0).stop 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 msgr2=0x7f7fa01a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa01a0b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7f98005660 tx=0x7f7f98005690 comp rx=0 tx=0).stop 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 shutdown_connections 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7fa0104bc0 0x7f7fa01a0b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f7f880afb80 0x7f7f880b2090 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa0103680 0x7f7fa01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 --2- 172.21.15.112:0/2563622627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa0104060 0x7f7fa0199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:14.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.514+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 >> 172.21.15.112:0/2563622627 conn(0x7f7fa00fded0 msgr2=0x7f7fa0107630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:14.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.515+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 shutdown_connections 2024-04-25T16:38:14.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:14.515+0000 7f7fa6f54700 1 -- 172.21.15.112:0/2563622627 wait complete. 2024-04-25T16:38:15.167 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T16:38:15.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:38:15.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/3316417374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78104d10 msgr2=0x7f2e78105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/3316417374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78104d10 0x7f2e78105120 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009900 tx=0x7f2e7402a410 comp rx=0 tx=0).stop 2024-04-25T16:38:15.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/3316417374 shutdown_connections 2024-04-25T16:38:15.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/3316417374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78109a10 0x7f2e7810be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/3316417374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78104d10 0x7f2e78105120 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/3316417374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78103ad0 0x7f2e781047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/3316417374 >> 172.21.15.112:0/3316417374 conn(0x7f2e780fecc0 msgr2=0x7f2e78101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/3316417374 shutdown_connections 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.292+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/3316417374 wait complete. 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 Processor -- start 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 -- start start 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 0x7f2e7819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78104d10 0x7f2e7819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78109a10 0x7f2e781a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7810f650 con 0x7f2e78103ad0 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7810f4d0 con 0x7f2e78109a10 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7fdf3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7810f7d0 con 0x7f2e78104d10 2024-04-25T16:38:15.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 0x7f2e7819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 0x7f2e7819a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53678/0 (socket says 172.21.15.112:53678) 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 -- 172.21.15.112:0/2320133132 learned_addr learned my addr 172.21.15.112:0/2320133132 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7e390640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78109a10 0x7f2e781a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7d38e640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78104d10 0x7f2e7819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 -- 172.21.15.112:0/2320133132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78104d10 msgr2=0x7f2e7819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78104d10 0x7f2e7819a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 -- 172.21.15.112:0/2320133132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78109a10 msgr2=0x7f2e781a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78109a10 0x7f2e781a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7db8f640 1 -- 172.21.15.112:0/2320133132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e781a1d10 con 0x7f2e78103ad0 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7e390640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78109a10 0x7f2e781a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.293+0000 7f2e7d38e640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78104d10 0x7f2e7819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:15.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.294+0000 7f2e7db8f640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 0x7f2e7819a280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c970 tx=0x7f2e6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.294+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e68007c70 con 0x7f2e78103ad0 2024-04-25T16:38:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.294+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e68007e10 con 0x7f2e78103ad0 2024-04-25T16:38:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.294+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e781a1fa0 con 0x7f2e78103ad0 2024-04-25T16:38:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.294+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e680187b0 con 0x7f2e78103ad0 2024-04-25T16:38:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.294+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e781a24e0 con 0x7f2e78103ad0 2024-04-25T16:38:15.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.295+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e78108820 con 0x7f2e78103ad0 2024-04-25T16:38:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.296+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f2e6800fab0 con 0x7f2e78103ad0 2024-04-25T16:38:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.297+0000 7f2e6effd640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e640afaf0 0x7f2e640b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.297+0000 7f2e7d38e640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e640afaf0 0x7f2e640b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.297+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f2e680def50 con 0x7f2e78103ad0 2024-04-25T16:38:15.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.297+0000 7f2e7d38e640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e640afaf0 0x7f2e640b2000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2e74006010 tx=0x7f2e74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:15.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.301+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e68053ff0 con 0x7f2e78103ad0 2024-04-25T16:38:15.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.481+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e7802dd20 con 0x7f2e78103ad0 2024-04-25T16:38:15.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.482+0000 7f2e6effd640 1 -- 172.21.15.112:0/2320133132 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22824 (secure 0 0 0) 0x7f2e68014030 con 0x7f2e78103ad0 2024-04-25T16:38:15.481 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:15.481 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":77,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:37:16.199786+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:38:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e640afaf0 msgr2=0x7f2e640b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e640afaf0 0x7f2e640b2000 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2e74006010 tx=0x7f2e74000bc0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 msgr2=0x7f2e7819a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 0x7f2e7819a280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2e6800c970 tx=0x7f2e6800ce30 comp rx=0 tx=0).stop 2024-04-25T16:38:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 shutdown_connections 2024-04-25T16:38:15.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2e640afaf0 0x7f2e640b2000 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e78104d10 0x7f2e7819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e78109a10 0x7f2e781a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 --2- 172.21.15.112:0/2320133132 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2e78103ad0 0x7f2e7819a280 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.484+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 >> 172.21.15.112:0/2320133132 conn(0x7f2e780fecc0 msgr2=0x7f2e781059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:15.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.485+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 shutdown_connections 2024-04-25T16:38:15.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.485+0000 7f2e7fdf3640 1 -- 172.21.15.112:0/2320133132 wait complete. 2024-04-25T16:38:15.493 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:15 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2563622627' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T16:38:15.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:36:24.204313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:36:25.525471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:38:15.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:38:15.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:15 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2563622627' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 -- 172.21.15.112:0/4167718855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638103af0 msgr2=0x7f16381047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 --2- 172.21.15.112:0/4167718855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638103af0 0x7f16381047f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1634009900 tx=0x7f163402a410 comp rx=0 tx=0).stop 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 -- 172.21.15.112:0/4167718855 shutdown_connections 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 --2- 172.21.15.112:0/4167718855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 0x7f1638105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 --2- 172.21.15.112:0/4167718855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638109a30 0x7f163810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 --2- 172.21.15.112:0/4167718855 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638103af0 0x7f16381047f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 -- 172.21.15.112:0/4167718855 >> 172.21.15.112:0/4167718855 conn(0x7f16380fece0 msgr2=0x7f1638101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 -- 172.21.15.112:0/4167718855 shutdown_connections 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.607+0000 7f1640357640 1 -- 172.21.15.112:0/4167718855 wait complete. 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 Processor -- start 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 -- start start 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638103af0 0x7f1638077960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 0x7f1638076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 0x7f1638076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f163810f5d0 con 0x7f1638109a30 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f163810f450 con 0x7f1638103af0 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f1640357640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f163810f750 con 0x7f1638104d30 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 0x7f1638076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 0x7f1638076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39832/0 (socket says 172.21.15.112:39832) 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 -- 172.21.15.112:0/489988514 learned_addr learned my addr 172.21.15.112:0/489988514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:15.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163d8f2640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 0x7f1638076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e0f3640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638103af0 0x7f1638077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 -- 172.21.15.112:0/489988514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 msgr2=0x7f1638076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 0x7f1638076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 -- 172.21.15.112:0/489988514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638103af0 msgr2=0x7f1638077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.608+0000 7f163e8f4640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638103af0 0x7f1638077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f163e8f4640 1 -- 172.21.15.112:0/489988514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1638076be0 con 0x7f1638109a30 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f163d8f2640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 0x7f1638076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f163e0f3640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638103af0 0x7f1638077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f163e8f4640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 0x7f1638076540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f163000eaa0 tx=0x7f163000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163000cdb0 con 0x7f1638109a30 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1630005020 con 0x7f1638109a30 2024-04-25T16:38:15.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16381a42f0 con 0x7f1638109a30 2024-04-25T16:38:15.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300116d0 con 0x7f1638109a30 2024-04-25T16:38:15.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.609+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16381a4700 con 0x7f1638109a30 2024-04-25T16:38:15.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.610+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1638108840 con 0x7f1638109a30 2024-04-25T16:38:15.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.611+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f16300051d0 con 0x7f1638109a30 2024-04-25T16:38:15.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.612+0000 7f162f7fe640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f16240afaf0 0x7f16240b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.612+0000 7f163e0f3640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f16240afaf0 0x7f16240b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.612+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7f16300df2b0 con 0x7f1638109a30 2024-04-25T16:38:15.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.612+0000 7f163e0f3640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f16240afaf0 0x7f16240b2000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1634009230 tx=0x7f1634000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:15.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.615+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1630016030 con 0x7f1638109a30 2024-04-25T16:38:15.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:15 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2563622627' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T16:38:15.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.792+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1638063840 con 0x7f1638109a30 2024-04-25T16:38:15.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.792+0000 7f162f7fe640 1 -- 172.21.15.112:0/489988514 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f16300a7e70 con 0x7f1638109a30 2024-04-25T16:38:15.791 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:38:15.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f16240afaf0 msgr2=0x7f16240b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f16240afaf0 0x7f16240b2000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1634009230 tx=0x7f1634000bc0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.793 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 msgr2=0x7f1638076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 0x7f1638076540 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f163000eaa0 tx=0x7f163000ef60 comp rx=0 tx=0).stop 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 shutdown_connections 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f16240afaf0 0x7f16240b2000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1638104d30 0x7f1638076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1638109a30 0x7f1638076540 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 --2- 172.21.15.112:0/489988514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1638103af0 0x7f1638077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 >> 172.21.15.112:0/489988514 conn(0x7f16380fece0 msgr2=0x7f1638105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 shutdown_connections 2024-04-25T16:38:15.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.794+0000 7f1640357640 1 -- 172.21.15.112:0/489988514 wait complete. 2024-04-25T16:38:15.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:38:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.916+0000 7fd24c2d8640 1 -- 172.21.15.112:0/182626308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441045f0 msgr2=0x7fd244106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.916+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/182626308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441045f0 0x7fd244106a10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd240009900 tx=0x7fd24002a410 comp rx=0 tx=0).stop 2024-04-25T16:38:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 -- 172.21.15.112:0/182626308 shutdown_connections 2024-04-25T16:38:15.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/182626308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2441098b0 0x7fd24410bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/182626308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd244106f50 0x7fd244109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/182626308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441045f0 0x7fd244106a10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 -- 172.21.15.112:0/182626308 >> 172.21.15.112:0/182626308 conn(0x7fd2440fdbf0 msgr2=0x7fd2441000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 -- 172.21.15.112:0/182626308 shutdown_connections 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 -- 172.21.15.112:0/182626308 wait complete. 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 Processor -- start 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 -- start start 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2441045f0 0x7fd244103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd244106f50 0x7fd2441020e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.917+0000 7fd24c2d8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 0x7fd244102620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24c2d8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410f600 con 0x7fd2441098b0 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24c2d8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410f480 con 0x7fd2441045f0 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24c2d8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd24410f780 con 0x7fd244106f50 2024-04-25T16:38:15.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 0x7fd244102620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a074640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2441045f0 0x7fd244103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 0x7fd244102620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39858/0 (socket says 172.21.15.112:39858) 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 -- 172.21.15.112:0/1688821364 learned_addr learned my addr 172.21.15.112:0/1688821364 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd249873640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd244106f50 0x7fd2441020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 -- 172.21.15.112:0/1688821364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd244106f50 msgr2=0x7fd2441020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd244106f50 0x7fd2441020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 -- 172.21.15.112:0/1688821364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2441045f0 msgr2=0x7fd244103a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2441045f0 0x7fd244103a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 -- 172.21.15.112:0/1688821364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244102d50 con 0x7fd2441098b0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24a875640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 0x7fd244102620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e9d0 tx=0x7fd23c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00cd80 con 0x7fd2441098b0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2441a4300 con 0x7fd2441098b0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd23c005020 con 0x7fd2441098b0 2024-04-25T16:38:15.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0116d0 con 0x7fd2441098b0 2024-04-25T16:38:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.918+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2441a47c0 con 0x7fd2441098b0 2024-04-25T16:38:15.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.919+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2440ff710 con 0x7fd2441098b0 2024-04-25T16:38:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.921+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fd23c0051d0 con 0x7fd2441098b0 2024-04-25T16:38:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.921+0000 7fd23b7fe640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2300afaf0 0x7fd2300b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.921+0000 7fd24a074640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2300afaf0 0x7fd2300b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.921+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8269+0+0 (secure 0 0 0) 0x7fd23c0df1c0 con 0x7fd2441098b0 2024-04-25T16:38:15.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.922+0000 7fd24a074640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2300afaf0 0x7fd2300b2000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd240009230 tx=0x7fd240031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:15.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:15.926+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd23c016030 con 0x7fd2441098b0 2024-04-25T16:38:16.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.102+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd2440637a0 con 0x7fd2441098b0 2024-04-25T16:38:16.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.103+0000 7fd23b7fe640 1 -- 172.21.15.112:0/1688821364 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fd23c0a7d80 con 0x7fd2441098b0 2024-04-25T16:38:16.101 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2300afaf0 msgr2=0x7fd2300b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2300afaf0 0x7fd2300b2000 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd240009230 tx=0x7fd240031400 comp rx=0 tx=0).stop 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 msgr2=0x7fd244102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 0x7fd244102620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e9d0 tx=0x7fd23c00ee90 comp rx=0 tx=0).stop 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 shutdown_connections 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fd2300afaf0 0x7fd2300b2000 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd244106f50 0x7fd2441020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2441098b0 0x7fd244102620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.104+0000 7fd24c2d8640 1 --2- 172.21.15.112:0/1688821364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2441045f0 0x7fd244103a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.105+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 >> 172.21.15.112:0/1688821364 conn(0x7fd2440fdbf0 msgr2=0x7fd244105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.105+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 shutdown_connections 2024-04-25T16:38:16.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.105+0000 7fd24c2d8640 1 -- 172.21.15.112:0/1688821364 wait complete. 2024-04-25T16:38:16.114 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:38:16.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.228+0000 7f6bfd346640 1 -- 172.21.15.112:0/1381519433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8105090 msgr2=0x7f6bf81054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.228+0000 7f6bfd346640 1 --2- 172.21.15.112:0/1381519433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8105090 0x7f6bf81054a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009900 tx=0x7f6bec02a410 comp rx=0 tx=0).stop 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 -- 172.21.15.112:0/1381519433 shutdown_connections 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 --2- 172.21.15.112:0/1381519433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8101740 0x7f6bf8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 --2- 172.21.15.112:0/1381519433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 0x7f6bf810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 --2- 172.21.15.112:0/1381519433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8105090 0x7f6bf81054a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 -- 172.21.15.112:0/1381519433 >> 172.21.15.112:0/1381519433 conn(0x7f6bf80fd340 msgr2=0x7f6bf80ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 -- 172.21.15.112:0/1381519433 shutdown_connections 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 -- 172.21.15.112:0/1381519433 wait complete. 2024-04-25T16:38:16.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.229+0000 7f6bfd346640 1 Processor -- start 2024-04-25T16:38:16.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 -- start start 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 0x7f6bf8104240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 0x7f6bf81028e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8105090 0x7f6bf8102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810f5e0 con 0x7f6bf8101740 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810f460 con 0x7f6bf8102240 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bfd346640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6bf810f760 con 0x7f6bf8105090 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 0x7f6bf8104240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 0x7f6bf8104240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39886/0 (socket says 172.21.15.112:39886) 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 -- 172.21.15.112:0/3773198968 learned_addr learned my addr 172.21.15.112:0/3773198968 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf67fc640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 0x7f6bf81028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf77fe640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8105090 0x7f6bf8102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 -- 172.21.15.112:0/3773198968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8105090 msgr2=0x7f6bf8102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8105090 0x7f6bf8102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 -- 172.21.15.112:0/3773198968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 msgr2=0x7f6bf81028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 0x7f6bf81028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 -- 172.21.15.112:0/3773198968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bf8103550 con 0x7f6bf8101740 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf67fc640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 0x7f6bf81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf77fe640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8105090 0x7f6bf8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bf6ffd640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 0x7f6bf8104240 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006010 tx=0x7f6bec031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.230+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec03f030 con 0x7f6bf8101740 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.231+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bf81a4300 con 0x7f6bf8101740 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.231+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bec003aa0 con 0x7f6bf8101740 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.231+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bec035070 con 0x7f6bf8101740 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.231+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bf81a4700 con 0x7f6bf8101740 2024-04-25T16:38:16.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.231+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bf80692e0 con 0x7f6bf8101740 2024-04-25T16:38:16.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.233+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f6bec042030 con 0x7f6bf8101740 2024-04-25T16:38:16.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.233+0000 7f6bdffff640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f6be00afaf0 0x7f6be00b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.234+0000 7f6bf67fc640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f6be00afaf0 0x7f6be00b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.234+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8298+0+0 (secure 0 0 0) 0x7f6bec0ff720 con 0x7f6bf8101740 2024-04-25T16:38:16.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.234+0000 7f6bf67fc640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f6be00afaf0 0x7f6be00b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8103be0 tx=0x7f6be8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:16.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.237+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bec0c82c0 con 0x7f6bf8101740 2024-04-25T16:38:16.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.417+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6bf802db50 con 0x7f6bf8101740 2024-04-25T16:38:16.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.418+0000 7f6bdffff640 1 -- 172.21.15.112:0/3773198968 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7f6bec0c7bc0 con 0x7f6bf8101740 2024-04-25T16:38:16.416 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T16:38:16.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f6be00afaf0 msgr2=0x7f6be00b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f6be00afaf0 0x7f6be00b2000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8103be0 tx=0x7f6be8005e70 comp rx=0 tx=0).stop 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 msgr2=0x7f6bf8104240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 0x7f6bf8104240 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006010 tx=0x7f6bec031a50 comp rx=0 tx=0).stop 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 shutdown_connections 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f6be00afaf0 0x7f6be00b2000 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6bf8105090 0x7f6bf8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bf8102240 0x7f6bf81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 --2- 172.21.15.112:0/3773198968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6bf8101740 0x7f6bf8104240 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 >> 172.21.15.112:0/3773198968 conn(0x7f6bf80fd340 msgr2=0x7f6bf8105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:16.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 shutdown_connections 2024-04-25T16:38:16.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.420+0000 7f6bfd346640 1 -- 172.21.15.112:0/3773198968 wait complete. 2024-04-25T16:38:16.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:16 smithi112 ceph-mon[29122]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:38:16.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:16 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T16:38:16.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:16 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2320133132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:16.429 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:16 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/489988514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:38:16.430 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:16 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1688821364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:38:16.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:38:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:16 smithi116 ceph-mon[32712]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:38:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:16 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T16:38:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:16 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2320133132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:16 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/489988514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:38:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:16 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1688821364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 -- 172.21.15.112:0/3168903337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc1033a0 msgr2=0x7f2bcc1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/3168903337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc1033a0 0x7f2bcc1037b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009900 tx=0x7f2bc002a410 comp rx=0 tx=0).stop 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 -- 172.21.15.112:0/3168903337 shutdown_connections 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/3168903337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bcc1080a0 0x7f2bcc10a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/3168903337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc1033a0 0x7f2bcc1037b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/3168903337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bcc102160 0x7f2bcc102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 -- 172.21.15.112:0/3168903337 >> 172.21.15.112:0/3168903337 conn(0x7f2bcc0fd330 msgr2=0x7f2bcc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 -- 172.21.15.112:0/3168903337 shutdown_connections 2024-04-25T16:38:16.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.542+0000 7f2bd26f2640 1 -- 172.21.15.112:0/3168903337 wait complete. 2024-04-25T16:38:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 Processor -- start 2024-04-25T16:38:16.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 -- start start 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 0x7f2bcc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bcc1033a0 0x7f2bcc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bcc1080a0 0x7f2bcc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10dc80 con 0x7f2bcc102160 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10db00 con 0x7f2bcc1080a0 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd26f2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bcc10de00 con 0x7f2bcc1033a0 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 0x7f2bcc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 0x7f2bcc199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39898/0 (socket says 172.21.15.112:39898) 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 -- 172.21.15.112:0/864894188 learned_addr learned my addr 172.21.15.112:0/864894188 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcb7fe640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bcc1033a0 0x7f2bcc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bd0c8f640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bcc1080a0 0x7f2bcc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 -- 172.21.15.112:0/864894188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bcc1033a0 msgr2=0x7f2bcc1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bcc1033a0 0x7f2bcc1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 -- 172.21.15.112:0/864894188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bcc1080a0 msgr2=0x7f2bcc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bcc1080a0 0x7f2bcc1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.543+0000 7f2bcbfff640 1 -- 172.21.15.112:0/864894188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc1a0cb0 con 0x7f2bcc102160 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.544+0000 7f2bcbfff640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 0x7f2bcc199170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc007920 tx=0x7f2bbc007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.544+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc019020 con 0x7f2bcc102160 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.544+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bbc00e6f0 con 0x7f2bcc102160 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.544+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc003870 con 0x7f2bcc102160 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.544+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc0fda00 con 0x7f2bcc102160 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.544+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bcc0fdf50 con 0x7f2bcc102160 2024-04-25T16:38:16.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.545+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bcc06a190 con 0x7f2bcc102160 2024-04-25T16:38:16.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.546+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f2bbc00e8a0 con 0x7f2bcc102160 2024-04-25T16:38:16.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.547+0000 7f2bc97fa640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2bb40afb70 0x7f2bb40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.547+0000 7f2bcb7fe640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2bb40afb70 0x7f2bb40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.547+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8298+0+0 (secure 0 0 0) 0x7f2bbc0e0570 con 0x7f2bcc102160 2024-04-25T16:38:16.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.548+0000 7f2bcb7fe640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2bb40afb70 0x7f2bb40b2080 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:16.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.551+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bbc023020 con 0x7f2bcc102160 2024-04-25T16:38:16.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:16 smithi133 ceph-mon[32691]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T16:38:16.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:16 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-04-25T16:38:16.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:16 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2320133132' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:16 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/489988514' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:38:16.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:16 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1688821364' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:38:16.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.768+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bcc107210 con 0x7f2bcc102160 2024-04-25T16:38:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.768+0000 7f2bc97fa640 1 -- 172.21.15.112:0/864894188 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11878 (secure 0 0 0) 0x7f2bbc0a9140 con 0x7f2bcc102160 2024-04-25T16:38:16.767 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:16.767 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:36:31.235238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24514,24524,24560]},"id":1}]} 2024-04-25T16:38:16.767 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2bb40afb70 msgr2=0x7f2bb40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2bb40afb70 0x7f2bb40b2080 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0000c00 tx=0x7f2bc0031400 comp rx=0 tx=0).stop 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 msgr2=0x7f2bcc199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 0x7f2bcc199170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc007920 tx=0x7f2bbc007de0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 shutdown_connections 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f2bb40afb70 0x7f2bb40b2080 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bcc1033a0 0x7f2bcc1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2bcc1080a0 0x7f2bcc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 --2- 172.21.15.112:0/864894188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2bcc102160 0x7f2bcc199170 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 >> 172.21.15.112:0/864894188 conn(0x7f2bcc0fd330 msgr2=0x7f2bcc100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 shutdown_connections 2024-04-25T16:38:16.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.770+0000 7f2bd26f2640 1 -- 172.21.15.112:0/864894188 wait complete. 2024-04-25T16:38:16.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:38:16.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 -- 172.21.15.112:0/1115421038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c105080 msgr2=0x7fca0c105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 --2- 172.21.15.112:0/1115421038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c105080 0x7fca0c105490 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca04009ca0 tx=0x7fca0402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 -- 172.21.15.112:0/1115421038 shutdown_connections 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 --2- 172.21.15.112:0/1115421038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca0c102230 0x7fca0c10bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 --2- 172.21.15.112:0/1115421038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 0x7fca0c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 --2- 172.21.15.112:0/1115421038 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c105080 0x7fca0c105490 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 -- 172.21.15.112:0/1115421038 >> 172.21.15.112:0/1115421038 conn(0x7fca0c0fd330 msgr2=0x7fca0c0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 -- 172.21.15.112:0/1115421038 shutdown_connections 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.890+0000 7fca13682640 1 -- 172.21.15.112:0/1115421038 wait complete. 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 Processor -- start 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 -- start start 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 0x7fca0c19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 0x7fca0c19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca0c105080 0x7fca0c1a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c10f5f0 con 0x7fca0c102230 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c10f470 con 0x7fca0c101730 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca13682640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fca0c10f770 con 0x7fca0c105080 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca10c1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 0x7fca0c19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca10c1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 0x7fca0c19a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39910/0 (socket says 172.21.15.112:39910) 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca10c1d640 1 -- 172.21.15.112:0/574401354 learned_addr learned my addr 172.21.15.112:0/574401354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.891+0000 7fca1141e640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 0x7fca0c19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca11c1f640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca0c105080 0x7fca0c1a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca10c1d640 1 -- 172.21.15.112:0/574401354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca0c105080 msgr2=0x7fca0c1a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca10c1d640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca0c105080 0x7fca0c1a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca10c1d640 1 -- 172.21.15.112:0/574401354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 msgr2=0x7fca0c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca10c1d640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 0x7fca0c19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca10c1d640 1 -- 172.21.15.112:0/574401354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0c1a1d90 con 0x7fca0c102230 2024-04-25T16:38:16.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca1141e640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 0x7fca0c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca10c1d640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 0x7fca0c19a790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d950 tx=0x7fca0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00017070 con 0x7fca0c102230 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca00009c30 con 0x7fca0c102230 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0c1a2080 con 0x7fca0c102230 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0000f620 con 0x7fca0c102230 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.892+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0c1a25d0 con 0x7fca0c102230 2024-04-25T16:38:16.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.893+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0c108a10 con 0x7fca0c102230 2024-04-25T16:38:16.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.894+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fca00010460 con 0x7fca0c102230 2024-04-25T16:38:16.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.895+0000 7fca0a7fc640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc9f40afaf0 0x7fc9f40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:16.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.895+0000 7fca1141e640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc9f40afaf0 0x7fc9f40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:16.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.895+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8298+0+0 (secure 0 0 0) 0x7fca000de9a0 con 0x7fca0c102230 2024-04-25T16:38:16.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.896+0000 7fca1141e640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc9f40afaf0 0x7fc9f40b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca04000f80 tx=0x7fca04004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:16.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:16.899+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca000a74e0 con 0x7fca0c102230 2024-04-25T16:38:17.078 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.080+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca0c02dd20 con 0x7fca0c102230 2024-04-25T16:38:17.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.081+0000 7fca0a7fc640 1 -- 172.21.15.112:0/574401354 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+22871 (secure 0 0 0) 0x7fca00009de0 con 0x7fca0c102230 2024-04-25T16:38:17.081 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:17.081 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":78,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:38:16.205563+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.083+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc9f40afaf0 msgr2=0x7fc9f40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.083+0000 7fca13682640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc9f40afaf0 0x7fc9f40b2000 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fca04000f80 tx=0x7fca04004a40 comp rx=0 tx=0).stop 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.083+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 msgr2=0x7fca0c19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.083+0000 7fca13682640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 0x7fca0c19a790 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fca0000d950 tx=0x7fca0000de10 comp rx=0 tx=0).stop 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 shutdown_connections 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc9f40afaf0 0x7fc9f40b2000 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca0c105080 0x7fca0c1a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fca0c102230 0x7fca0c19a790 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 --2- 172.21.15.112:0/574401354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca0c101730 0x7fca0c19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 >> 172.21.15.112:0/574401354 conn(0x7fca0c0fd330 msgr2=0x7fca0c105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:17.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 shutdown_connections 2024-04-25T16:38:17.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.084+0000 7fca13682640 1 -- 172.21.15.112:0/574401354 wait complete. 2024-04-25T16:38:17.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:36:24.204313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:36:25.525471+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:38:17.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 -- 172.21.15.112:0/2943844758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 msgr2=0x7f976410ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 --2- 172.21.15.112:0/2943844758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410ad40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f975c009ca0 tx=0x7f975c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 -- 172.21.15.112:0/2943844758 shutdown_connections 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 --2- 172.21.15.112:0/2943844758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9764103c20 0x7f9764104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 --2- 172.21.15.112:0/2943844758 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410ad40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 --2- 172.21.15.112:0/2943844758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641029e0 0x7f97641036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 -- 172.21.15.112:0/2943844758 >> 172.21.15.112:0/2943844758 conn(0x7f97640fdbd0 msgr2=0x7f9764100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 -- 172.21.15.112:0/2943844758 shutdown_connections 2024-04-25T16:38:17.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.206+0000 7f976b19e640 1 -- 172.21.15.112:0/2943844758 wait complete. 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 Processor -- start 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 -- start start 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641029e0 0x7f976410d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9764103c20 0x7f976410b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f976410f610 con 0x7f9764108920 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f976410f490 con 0x7f97641029e0 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976b19e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f976410f790 con 0x7f9764103c20 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39916/0 (socket says 172.21.15.112:39916) 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 -- 172.21.15.112:0/2898112882 learned_addr learned my addr 172.21.15.112:0/2898112882 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 -- 172.21.15.112:0/2898112882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9764103c20 msgr2=0x7f976410b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9764103c20 0x7f976410b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f9768f3a640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641029e0 0x7f976410d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 -- 172.21.15.112:0/2898112882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641029e0 msgr2=0x7f976410d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641029e0 0x7f976410d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 -- 172.21.15.112:0/2898112882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976410c5c0 con 0x7f9764108920 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.207+0000 7f976973b640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410be90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f975c006010 tx=0x7f975c02f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.208+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c003c70 con 0x7f9764108920 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.208+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c003e10 con 0x7f9764108920 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.208+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97641a42d0 con 0x7f9764108920 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.208+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c02fcc0 con 0x7f9764108920 2024-04-25T16:38:17.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.208+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a4790 con 0x7f9764108920 2024-04-25T16:38:17.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.209+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764107730 con 0x7f9764108920 2024-04-25T16:38:17.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.210+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f975c004710 con 0x7f9764108920 2024-04-25T16:38:17.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.211+0000 7f9761ffb640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f97540afb70 0x7f97540b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.211+0000 7f9768f3a640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f97540afb70 0x7f97540b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.211+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8298+0+0 (secure 0 0 0) 0x7f975c100620 con 0x7f9764108920 2024-04-25T16:38:17.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.211+0000 7f9768f3a640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f97540afb70 0x7f97540b2080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9758005fd0 tx=0x7f9758005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:17.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.215+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f975c0c91c0 con 0x7f9764108920 2024-04-25T16:38:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:17 smithi112 ceph-mon[29122]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T16:38:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:17 smithi112 ceph-mon[29122]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T16:38:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:17 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3773198968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:38:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:17 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/864894188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:38:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:17 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/574401354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:17.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.396+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97640637a0 con 0x7f9764108920 2024-04-25T16:38:17.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.396+0000 7f9761ffb640 1 -- 172.21.15.112:0/2898112882 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f975c0c8ac0 con 0x7f9764108920 2024-04-25T16:38:17.395 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:38:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f97540afb70 msgr2=0x7f97540b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f97540afb70 0x7f97540b2080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9758005fd0 tx=0x7f9758005850 comp rx=0 tx=0).stop 2024-04-25T16:38:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 msgr2=0x7f976410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410be90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f975c006010 tx=0x7f975c02f3e0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 shutdown_connections 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f97540afb70 0x7f97540b2080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.398+0000 7f976b19e640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9764103c20 0x7f976410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.399+0000 7f976b19e640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9764108920 0x7f976410be90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.399+0000 7f976b19e640 1 --2- 172.21.15.112:0/2898112882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97641029e0 0x7f976410d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.399+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 >> 172.21.15.112:0/2898112882 conn(0x7f97640fdbd0 msgr2=0x7f9764104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.399+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 shutdown_connections 2024-04-25T16:38:17.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.399+0000 7f976b19e640 1 -- 172.21.15.112:0/2898112882 wait complete. 2024-04-25T16:38:17.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:38:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:17 smithi116 ceph-mon[32712]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T16:38:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:17 smithi116 ceph-mon[32712]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T16:38:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:17 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3773198968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:38:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:17 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/864894188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:38:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:17 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/574401354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 -- 172.21.15.112:0/4184487390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0103af0 msgr2=0x7fdff01047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 --2- 172.21.15.112:0/4184487390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0103af0 0x7fdff01047f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009900 tx=0x7fdfe402a410 comp rx=0 tx=0).stop 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 -- 172.21.15.112:0/4184487390 shutdown_connections 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 --2- 172.21.15.112:0/4184487390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0109a30 0x7fdff010be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 --2- 172.21.15.112:0/4184487390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdff0104d30 0x7fdff0105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 --2- 172.21.15.112:0/4184487390 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0103af0 0x7fdff01047f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 -- 172.21.15.112:0/4184487390 >> 172.21.15.112:0/4184487390 conn(0x7fdff00fece0 msgr2=0x7fdff0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:17.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 -- 172.21.15.112:0/4184487390 shutdown_connections 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.520+0000 7fdff6c40640 1 -- 172.21.15.112:0/4184487390 wait complete. 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 Processor -- start 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 -- start start 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0103af0 0x7fdff019a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 0x7fdff019a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdff0109a30 0x7fdff01a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdff010f650 con 0x7fdff0104d30 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdff010f4d0 con 0x7fdff0109a30 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff6c40640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdff010f7d0 con 0x7fdff0103af0 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdfeffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 0x7fdff019a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdfeffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 0x7fdff019a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39936/0 (socket says 172.21.15.112:39936) 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdfeffff640 1 -- 172.21.15.112:0/3166979598 learned_addr learned my addr 172.21.15.112:0/3166979598 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:17.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff51dd640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdff0109a30 0x7fdff01a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.521+0000 7fdff49dc640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0103af0 0x7fdff019a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfeffff640 1 -- 172.21.15.112:0/3166979598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0103af0 msgr2=0x7fdff019a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfeffff640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0103af0 0x7fdff019a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfeffff640 1 -- 172.21.15.112:0/3166979598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdff0109a30 msgr2=0x7fdff01a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfeffff640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdff0109a30 0x7fdff01a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfeffff640 1 -- 172.21.15.112:0/3166979598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff01a1db0 con 0x7fdff0104d30 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdff49dc640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0103af0 0x7fdff019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfeffff640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 0x7fdff019a820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc017070 con 0x7fdff0104d30 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff01a20a0 con 0x7fdff0104d30 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc009c30 con 0x7fdff0104d30 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00f620 con 0x7fdff0104d30 2024-04-25T16:38:17.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.522+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01a26c0 con 0x7fdff0104d30 2024-04-25T16:38:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.523+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff0108840 con 0x7fdff0104d30 2024-04-25T16:38:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.524+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fdfdc010460 con 0x7fdff0104d30 2024-04-25T16:38:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.525+0000 7fdfedffb640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdfe00afb70 0x7fdfe00b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.525+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fdfdc0de800 con 0x7fdff0104d30 2024-04-25T16:38:17.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.525+0000 7fdff49dc640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdfe00afb70 0x7fdfe00b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.526+0000 7fdff49dc640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdfe00afb70 0x7fdfe00b2080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005f70 tx=0x7fdfe4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:17.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.529+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfdc0a7420 con 0x7fdff0104d30 2024-04-25T16:38:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:17 smithi133 ceph-mon[32691]: from='mgr.14152 172.21.15.112:0/3704884434' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-04-25T16:38:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:17 smithi133 ceph-mon[32691]: osdmap e78: 12 total, 12 up, 12 in 2024-04-25T16:38:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:17 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3773198968' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:38:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:17 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/864894188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:38:17.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:17 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/574401354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.709+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdff00637a0 con 0x7fdff0104d30 2024-04-25T16:38:17.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.709+0000 7fdfedffb640 1 -- 172.21.15.112:0/3166979598 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7fdfdc009de0 con 0x7fdff0104d30 2024-04-25T16:38:17.708 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdfe00afb70 msgr2=0x7fdfe00b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdfe00afb70 0x7fdfe00b2080 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4005f70 tx=0x7fdfe4000bc0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 msgr2=0x7fdff019a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 0x7fdff019a820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00de10 comp rx=0 tx=0).stop 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 shutdown_connections 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdff0109a30 0x7fdff01a1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdff0104d30 0x7fdff019a820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fdfe00afb70 0x7fdfe00b2080 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 --2- 172.21.15.112:0/3166979598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdff0103af0 0x7fdff019a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 >> 172.21.15.112:0/3166979598 conn(0x7fdff00fece0 msgr2=0x7fdff0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:17.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 shutdown_connections 2024-04-25T16:38:17.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.711+0000 7fdff6c40640 1 -- 172.21.15.112:0/3166979598 wait complete. 2024-04-25T16:38:17.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 -- 172.21.15.112:0/796193579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc670108a20 msgr2=0x7fc67010ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 --2- 172.21.15.112:0/796193579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc670108a20 0x7fc67010ae40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc668009ca0 tx=0x7fc66802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 -- 172.21.15.112:0/796193579 shutdown_connections 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 --2- 172.21.15.112:0/796193579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6701043d0 0x7fc670102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 --2- 172.21.15.112:0/796193579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc670108a20 0x7fc67010ae40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 --2- 172.21.15.112:0/796193579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc670103ef0 0x7fc6701027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.833+0000 7fc676384640 1 -- 172.21.15.112:0/796193579 >> 172.21.15.112:0/796193579 conn(0x7fc6700fd350 msgr2=0x7fc6700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 -- 172.21.15.112:0/796193579 shutdown_connections 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 -- 172.21.15.112:0/796193579 wait complete. 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 Processor -- start 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 -- start start 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc670103ef0 0x7fc670198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc670108a20 0x7fc670199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 0x7fc6701a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc67010e530 con 0x7fc67019e1a0 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc67010e3b0 con 0x7fc670108a20 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.834+0000 7fc676384640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc67010e6b0 con 0x7fc670103ef0 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 0x7fc6701a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 0x7fc6701a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39966/0 (socket says 172.21.15.112:39966) 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 -- 172.21.15.112:0/1717486658 learned_addr learned my addr 172.21.15.112:0/1717486658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc66ffff640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc670103ef0 0x7fc670198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc66f7fe640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc670108a20 0x7fc670199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 -- 172.21.15.112:0/1717486658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc670103ef0 msgr2=0x7fc670198cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc670103ef0 0x7fc670198cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 -- 172.21.15.112:0/1717486658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc670108a20 msgr2=0x7fc670199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc670108a20 0x7fc670199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 -- 172.21.15.112:0/1717486658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6701a0c30 con 0x7fc67019e1a0 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc674921640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 0x7fc6701a05c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc6680097b0 tx=0x7fc668031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc668003bf0 con 0x7fc67019e1a0 2024-04-25T16:38:17.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6701a0ec0 con 0x7fc67019e1a0 2024-04-25T16:38:17.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc668003d90 con 0x7fc67019e1a0 2024-04-25T16:38:17.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66802fc30 con 0x7fc67019e1a0 2024-04-25T16:38:17.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.835+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6701a1330 con 0x7fc67019e1a0 2024-04-25T16:38:17.838 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.836+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc670107120 con 0x7fc67019e1a0 2024-04-25T16:38:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.837+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fc668004040 con 0x7fc67019e1a0 2024-04-25T16:38:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.838+0000 7fc66d7fa640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6580afb20 0x7fc6580b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.838+0000 7fc66ffff640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6580afb20 0x7fc6580b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.839+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fc668100360 con 0x7fc67019e1a0 2024-04-25T16:38:17.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.839+0000 7fc66ffff640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6580afb20 0x7fc6580b2030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc670199b80 tx=0x7fc660005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:17.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:17.843+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6680c8f00 con 0x7fc67019e1a0 2024-04-25T16:38:18.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.022+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc67002db50 con 0x7fc67019e1a0 2024-04-25T16:38:18.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.022+0000 7fc66d7fa640 1 -- 172.21.15.112:0/1717486658 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7fc6680c8800 con 0x7fc67019e1a0 2024-04-25T16:38:18.021 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6580afb20 msgr2=0x7fc6580b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6580afb20 0x7fc6580b2030 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc670199b80 tx=0x7fc660005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 msgr2=0x7fc6701a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 0x7fc6701a05c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc6680097b0 tx=0x7fc668031840 comp rx=0 tx=0).stop 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 shutdown_connections 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc67019e1a0 0x7fc6701a05c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc670108a20 0x7fc670199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fc6580afb20 0x7fc6580b2030 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 --2- 172.21.15.112:0/1717486658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc670103ef0 0x7fc670198cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 >> 172.21.15.112:0/1717486658 conn(0x7fc6700fd350 msgr2=0x7fc6700ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.024+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 shutdown_connections 2024-04-25T16:38:18.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.025+0000 7fc676384640 1 -- 172.21.15.112:0/1717486658 wait complete. 2024-04-25T16:38:18.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:38:18.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 -- 172.21.15.112:0/3253246279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 msgr2=0x7f4054101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3253246279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f4054101bc0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4040009900 tx=0x7f404002a410 comp rx=0 tx=0).stop 2024-04-25T16:38:18.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 -- 172.21.15.112:0/3253246279 shutdown_connections 2024-04-25T16:38:18.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3253246279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 0x7f405410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3253246279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f4054101bc0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3253246279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 0x7f40541054b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 -- 172.21.15.112:0/3253246279 >> 172.21.15.112:0/3253246279 conn(0x7f40540fd370 msgr2=0x7f40540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 -- 172.21.15.112:0/3253246279 shutdown_connections 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.143+0000 7f405aeb6640 1 -- 172.21.15.112:0/3253246279 wait complete. 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 Processor -- start 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 -- start start 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f405419a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 0x7f405419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 0x7f40541a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f405410f610 con 0x7f4054101750 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f405410f490 con 0x7f4054102250 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f405aeb6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f405410f790 con 0x7f40541050a0 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f4058c52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f405419a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f4058c52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f405419a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39984/0 (socket says 172.21.15.112:39984) 2024-04-25T16:38:18.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f4058c52640 1 -- 172.21.15.112:0/3723527923 learned_addr learned my addr 172.21.15.112:0/3723527923 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f4053fff640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 0x7f405419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f4058c52640 1 -- 172.21.15.112:0/3723527923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 msgr2=0x7f40541a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.144+0000 7f4059453640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 0x7f40541a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4058c52640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 0x7f40541a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4058c52640 1 -- 172.21.15.112:0/3723527923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 msgr2=0x7f405419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4058c52640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 0x7f405419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4058c52640 1 -- 172.21.15.112:0/3723527923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40541a1be0 con 0x7f4054101750 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4053fff640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 0x7f405419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4059453640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 0x7f40541a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4058c52640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f405419a150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f404800c970 tx=0x7f404800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4048007c70 con 0x7f4054101750 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4048007e10 con 0x7f4054101750 2024-04-25T16:38:18.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40541a1ed0 con 0x7f4054101750 2024-04-25T16:38:18.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40480187b0 con 0x7f4054101750 2024-04-25T16:38:18.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.145+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40541a2330 con 0x7f4054101750 2024-04-25T16:38:18.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.146+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f405406a190 con 0x7f4054101750 2024-04-25T16:38:18.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.147+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f404800fab0 con 0x7f4054101750 2024-04-25T16:38:18.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.148+0000 7f4051ffb640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f40440afaf0 0x7f40440b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.148+0000 7f4053fff640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f40440afaf0 0x7f40440b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.148+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f40480ded50 con 0x7f4054101750 2024-04-25T16:38:18.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.149+0000 7f4053fff640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f40440afaf0 0x7f40440b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4040000c00 tx=0x7f4040005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.153+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40480e2030 con 0x7f4054101750 2024-04-25T16:38:18.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:18 smithi112 ceph-mon[29122]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T16:38:18.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:18 smithi112 ceph-mon[29122]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T16:38:18.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:18 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2898112882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:18 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3166979598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:18 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1717486658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.370+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40540637a0 con 0x7f4054101750 2024-04-25T16:38:18.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.371+0000 7f4051ffb640 1 -- 172.21.15.112:0/3723527923 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11878 (secure 0 0 0) 0x7f4048014030 con 0x7f4054101750 2024-04-25T16:38:18.370 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:18.370 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2123343031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2123343031},{"type":"v1","addr":"172.21.15.116:6833","nonce":2123343031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:36:31.235238+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.116:6835/4202555064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":4202555064},{"type":"v1","addr":"172.21.15.116:6835","nonce":4202555064}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24514,24524,24560]},"id":1}]} 2024-04-25T16:38:18.370 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 21 2024-04-25T16:38:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f40440afaf0 msgr2=0x7f40440b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f40440afaf0 0x7f40440b2000 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4040000c00 tx=0x7f4040005d50 comp rx=0 tx=0).stop 2024-04-25T16:38:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 msgr2=0x7f405419a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f405419a150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f404800c970 tx=0x7f404800ce30 comp rx=0 tx=0).stop 2024-04-25T16:38:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 shutdown_connections 2024-04-25T16:38:18.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f40440afaf0 0x7f40440b2000 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40541050a0 0x7f40541a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4054102250 0x7f405419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 --2- 172.21.15.112:0/3723527923 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4054101750 0x7f405419a150 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 >> 172.21.15.112:0/3723527923 conn(0x7f40540fd370 msgr2=0x7f4054105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.373+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 shutdown_connections 2024-04-25T16:38:18.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.374+0000 7f405aeb6640 1 -- 172.21.15.112:0/3723527923 wait complete. 2024-04-25T16:38:18.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:38:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 -- 172.21.15.112:0/1315573498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4101e60 msgr2=0x7fa8c4106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/1315573498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4101e60 0x7fa8c4106b20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009900 tx=0x7fa8b802a410 comp rx=0 tx=0).stop 2024-04-25T16:38:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 -- 172.21.15.112:0/1315573498 shutdown_connections 2024-04-25T16:38:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/1315573498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8c4102360 0x7fa8c41090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/1315573498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4109780 0x7fa8c410bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/1315573498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4101e60 0x7fa8c4106b20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 -- 172.21.15.112:0/1315573498 >> 172.21.15.112:0/1315573498 conn(0x7fa8c40fd3b0 msgr2=0x7fa8c40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 -- 172.21.15.112:0/1315573498 shutdown_connections 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.493+0000 7fa8cabb2640 1 -- 172.21.15.112:0/1315573498 wait complete. 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 Processor -- start 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 -- start start 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4101e60 0x7fa8c419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8c4102360 0x7fa8c419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 0x7fa8c41a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410f5e0 con 0x7fa8c4109780 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410f460 con 0x7fa8c4101e60 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8cabb2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410f760 con 0x7fa8c4102360 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 0x7fa8c41a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 0x7fa8c41a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39996/0 (socket says 172.21.15.112:39996) 2024-04-25T16:38:18.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 -- 172.21.15.112:0/2682341072 learned_addr learned my addr 172.21.15.112:0/2682341072 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c894e640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4101e60 0x7fa8c419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c3fff640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8c4102360 0x7fa8c419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 -- 172.21.15.112:0/2682341072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8c4102360 msgr2=0x7fa8c419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8c4102360 0x7fa8c419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 -- 172.21.15.112:0/2682341072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4101e60 msgr2=0x7fa8c419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4101e60 0x7fa8c419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 -- 172.21.15.112:0/2682341072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41a1d20 con 0x7fa8c4109780 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c894e640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4101e60 0x7fa8c419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.494+0000 7fa8c914f640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 0x7fa8c41a16b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00eaa0 tx=0x7fa8bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.495+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00cdb0 con 0x7fa8c4109780 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.495+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8bc005020 con 0x7fa8c4109780 2024-04-25T16:38:18.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.495+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0116d0 con 0x7fa8c4109780 2024-04-25T16:38:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.495+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41a1fb0 con 0x7fa8c4109780 2024-04-25T16:38:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.495+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a2470 con 0x7fa8c4109780 2024-04-25T16:38:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.496+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c40feed0 con 0x7fa8c4109780 2024-04-25T16:38:18.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.497+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fa8bc0051d0 con 0x7fa8c4109780 2024-04-25T16:38:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.498+0000 7fa8c1ffb640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa8b40afb70 0x7fa8b40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.498+0000 7fa8c894e640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa8b40afb70 0x7fa8b40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.498+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fa8bc0dee90 con 0x7fa8c4109780 2024-04-25T16:38:18.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.498+0000 7fa8c894e640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa8b40afb70 0x7fa8b40b2080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031440 tx=0x7fa8b8005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.503+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8bc016030 con 0x7fa8c4109780 2024-04-25T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:18 smithi116 ceph-mon[32712]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:18 smithi116 ceph-mon[32712]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:18 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2898112882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:18 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3166979598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:18 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1717486658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.680+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8c402dd70 con 0x7fa8c4109780 2024-04-25T16:38:18.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.681+0000 7fa8c1ffb640 1 -- 172.21.15.112:0/2682341072 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22871 (secure 0 0 0) 0x7fa8bc0a7a30 con 0x7fa8c4109780 2024-04-25T16:38:18.680 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:38:18.680 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":79,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:38:17.216469+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:38:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.683+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa8b40afb70 msgr2=0x7fa8b40b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa8b40afb70 0x7fa8b40b2080 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031440 tx=0x7fa8b8005d50 comp rx=0 tx=0).stop 2024-04-25T16:38:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 msgr2=0x7fa8c41a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 0x7fa8c41a16b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00eaa0 tx=0x7fa8bc00ef60 comp rx=0 tx=0).stop 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 shutdown_connections 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa8b40afb70 0x7fa8b40b2080 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8c4102360 0x7fa8c419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa8c4109780 0x7fa8c41a16b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 --2- 172.21.15.112:0/2682341072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8c4101e60 0x7fa8c419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 >> 172.21.15.112:0/2682341072 conn(0x7fa8c40fd3b0 msgr2=0x7fa8c4105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 shutdown_connections 2024-04-25T16:38:18.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.684+0000 7fa8cabb2640 1 -- 172.21.15.112:0/2682341072 wait complete. 2024-04-25T16:38:18.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T16:36:26.563199+0000', 'modified': '2024-04-25T16:36:31.235238+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24560, 'mds_1': 24524, 'mds_2': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24472': {'gid': 24472, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6835/2304762610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2304762610}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2304762610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24484': {'gid': 24484, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.133:6837/3840802165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6836', 'nonce': 3840802165}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3840802165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.116:6835/4202555064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 4202555064}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 4202555064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.133:6839/99930432', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6838', 'nonce': 99930432}, {'type': 'v1', 'addr': '172.21.15.133:6839', 'nonce': 99930432}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'h', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.116:6837/797562383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6836', 'nonce': 797562383}, {'type': 'v1', 'addr': '172.21.15.116:6837', 'nonce': 797562383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24560': {'gid': 24560, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6839/1016899540', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6838', 'nonce': 1016899540}, {'type': 'v1', 'addr': '172.21.15.116:6839', 'nonce': 1016899540}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24560, 'qdb_cluster': [24514, 24524, 24560]} 2024-04-25T16:38:18.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T16:38:18.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T16:38:18.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:18 smithi133 ceph-mon[32691]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T16:38:18.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:18 smithi133 ceph-mon[32691]: osdmap e79: 12 total, 12 up, 12 in 2024-04-25T16:38:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:18 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2898112882' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:18 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3166979598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:18 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1717486658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:38:18.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 -- 172.21.15.112:0/1724752259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0108920 msgr2=0x7f66e010ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1724752259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0108920 0x7f66e010ad40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009ca0 tx=0x7f66d802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 -- 172.21.15.112:0/1724752259 shutdown_connections 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1724752259 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0108920 0x7f66e010ad40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1724752259 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e0103c20 0x7f66e0104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1724752259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e01029e0 0x7f66e01036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 -- 172.21.15.112:0/1724752259 >> 172.21.15.112:0/1724752259 conn(0x7f66e00fdbd0 msgr2=0x7f66e0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 -- 172.21.15.112:0/1724752259 shutdown_connections 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 -- 172.21.15.112:0/1724752259 wait complete. 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 Processor -- start 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.805+0000 7f66e66e0640 1 -- start start 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e01029e0 0x7f66e019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 0x7f66e019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e0108920 0x7f66e01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010e4f0 con 0x7f66e0103c20 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010e370 con 0x7f66e01029e0 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010e670 con 0x7f66e0108920 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 0x7f66e019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 0x7f66e019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40006/0 (socket says 172.21.15.112:40006) 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 -- 172.21.15.112:0/1744300418 learned_addr learned my addr 172.21.15.112:0/1744300418 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66dffff640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e01029e0 0x7f66e019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 -- 172.21.15.112:0/1744300418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e0108920 msgr2=0x7f66e01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e0108920 0x7f66e01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e0108920 0x7f66e01a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 -- 172.21.15.112:0/1744300418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e01029e0 msgr2=0x7f66e019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e01029e0 0x7f66e019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 -- 172.21.15.112:0/1744300418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e01a2020 con 0x7f66e0103c20 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66dffff640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e01029e0 0x7f66e019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:18.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 0x7f66e019aa20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f66d000d950 tx=0x7f66d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66dd7fa640 1 -- 172.21.15.112:0/1744300418 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0017070 con 0x7f66e0103c20 2024-04-25T16:38:18.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66dd7fa640 1 -- 172.21.15.112:0/1744300418 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d0009c30 con 0x7f66e0103c20 2024-04-25T16:38:18.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e01a2310 con 0x7f66e0103c20 2024-04-25T16:38:18.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.806+0000 7f66dd7fa640 1 -- 172.21.15.112:0/1744300418 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d000f620 con 0x7f66e0103c20 2024-04-25T16:38:18.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.807+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01a2860 con 0x7f66e0103c20 2024-04-25T16:38:18.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66dd7fa640 1 -- 172.21.15.112:0/1744300418 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f66d0010460 con 0x7f66e0103c20 2024-04-25T16:38:18.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 Processor -- start 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 -- start start 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b5fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b2313b0 0x560a4b5fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b6027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a4b560740 con 0x560a4b239590 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a4b4fc3a0 con 0x560a4b25a270 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66ed58b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a4b55c700 con 0x560a4b2313b0 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66dd7fa640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f66c80afaf0 0x7f66c80b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b6027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b6027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40022/0 (socket says 172.21.15.112:40022) 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.809+0000 7f66df7fe640 1 -- 172.21.15.112:0/1644223309 learned_addr learned my addr 172.21.15.112:0/1644223309 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66dffff640 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b2313b0 0x560a4b5fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f66c80afaf0 0x7f66c80b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b5fb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66dd7fa640 1 -- 172.21.15.112:0/1744300418 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f66d00de970 con 0x7f66e0103c20 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66df7fe640 1 -- 172.21.15.112:0/1644223309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b2313b0 msgr2=0x560a4b5fb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b2313b0 0x560a4b5fb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66df7fe640 1 -- 172.21.15.112:0/1644223309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 msgr2=0x560a4b5fb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b5fb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66df7fe640 1 -- 172.21.15.112:0/1644223309 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4b518750 con 0x560a4b239590 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b5fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b6027f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f66d00a2e90 tx=0x7f66d0010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5913080bfc9fd96f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f66c80afaf0 0x7f66c80b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66d8004af0 tx=0x7f66d8004a60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66c67fc640 1 -- 172.21.15.112:0/1644223309 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0020070 con 0x560a4b239590 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66c67fc640 1 -- 172.21.15.112:0/1644223309 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d00a6920 con 0x560a4b239590 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.810+0000 7f66c67fc640 1 -- 172.21.15.112:0/1644223309 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00af9e0 con 0x560a4b239590 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 -- 172.21.15.112:0/1644223309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 msgr2=0x560a4b6027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b6027f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f66d00a2e90 tx=0x7f66d0010cb0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 -- 172.21.15.112:0/1644223309 shutdown_connections 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b2313b0 0x560a4b5fb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b6027f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1644223309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b5fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 -- 172.21.15.112:0/1644223309 >> 172.21.15.112:0/1644223309 conn(0x560a4b24e3c0 msgr2=0x560a4b5f6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 -- 172.21.15.112:0/1644223309 shutdown_connections 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 -- 172.21.15.112:0/1644223309 wait complete. 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.811+0000 7f66ed58b740 1 -- start start 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 Processor -- start 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b2313b0 0x560a4b6052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b605830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a4b6031e0 con 0x560a4b239590 2024-04-25T16:38:18.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a4b4e7fe0 con 0x560a4b2313b0 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a4b603360 con 0x560a4b25a270 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66ed58b740 10 client.0 _ll_register_callbacks cb 0x560a4b4fe150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b605830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b605830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40034/0 (socket says 172.21.15.112:40034) 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 -- 172.21.15.112:0/1215226970 learned_addr learned my addr 172.21.15.112:0/1215226970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66dffff640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b2313b0 0x560a4b6052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 -- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 msgr2=0x560a4b60c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 -- 172.21.15.112:0/1215226970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b2313b0 msgr2=0x560a4b6052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b2313b0 0x560a4b6052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 -- 172.21.15.112:0/1215226970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4b2722d0 con 0x560a4b239590 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66dffff640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b2313b0 0x560a4b6052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:18.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b605830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f66d00a7470 tx=0x7f66d00a3330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178f25d68fbe0d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.812+0000 7f66bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1215226970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0020040 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1215226970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f66d00a6070 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d00a6920 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1215226970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f66d00a7510 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 10 client.15228 fetch_fsmap learned FSMap version 21 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560a4b33c6b0 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1215226970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d00a6920 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1215226970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f66d00a5ae0 con 0x560a4b239590 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 10 client.15228 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 10 client.15228 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 20 client.15228 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b218e40 0x560a4b612a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 4 client.15228 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:38:18.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 --> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a4b56e350 con 0x560a4b218e40 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.813+0000 7f66dffff640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b218e40 0x560a4b612a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.814+0000 7f66dffff640 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b218e40 0x560a4b612a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e052630f008cfab server_cookie=5fa167043c607d7e in_seq=0 out_seq=0 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.814+0000 7f66bf7fe640 10 client.15228 ms_handle_connect on v2:172.21.15.133:6838/99930432 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1215226970 <== mds.2 v2:172.21.15.133:6838/99930432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a4b56e350 con 0x560a4b218e40 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66bf7fe640 10 client.15228 handle_command_reply: tid=0 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 client.15228 shutdown 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 msgr2=0x560a4b605830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b605830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f66d00a7470 tx=0x7f66d00a3330 comp rx=0 tx=0).stop 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 shutdown_connections 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b218e40 0x560a4b612a60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b239590 0x560a4b605830 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66bf7fe640 0 client.15228 ms_handle_reset on v2:172.21.15.133:6838/99930432 2024-04-25T16:38:18.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1215226970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b2313b0 0x560a4b6052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 >> 172.21.15.112:0/1215226970 conn(0x560a4b24e3c0 msgr2=0x560a4b5f5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 shutdown_connections 2024-04-25T16:38:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 1 -- 172.21.15.112:0/1215226970 wait complete. 2024-04-25T16:38:18.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.815+0000 7f66ed58b740 20 client.15228 trim_cache size 0 max 16384 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.833+0000 7f66ed58b740 1 Processor -- start 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 -- start start 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b5f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b239590 0x560a4b603910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b603e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a4b610da0 con 0x560a4b2313b0 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a4b611470 con 0x560a4b25a270 2024-04-25T16:38:18.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66ed58b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a4b611b40 con 0x560a4b239590 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b5f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b5f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40046/0 (socket says 172.21.15.112:40046) 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 -- 172.21.15.112:0/3497111565 learned_addr learned my addr 172.21.15.112:0/3497111565 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 -- 172.21.15.112:0/3497111565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b239590 msgr2=0x560a4b603910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b239590 0x560a4b603910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 -- 172.21.15.112:0/3497111565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 msgr2=0x560a4b603e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b603e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 -- 172.21.15.112:0/3497111565 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4b5f2430 con 0x560a4b2313b0 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66dffff640 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b5f1a20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009fd0 tx=0x7f66d4014a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6b760e9e7c4ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66c5ffb640 1 -- 172.21.15.112:0/3497111565 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4005020 con 0x560a4b2313b0 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66c5ffb640 1 -- 172.21.15.112:0/3497111565 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d4004730 con 0x560a4b2313b0 2024-04-25T16:38:18.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.834+0000 7f66c5ffb640 1 -- 172.21.15.112:0/3497111565 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d401b590 con 0x560a4b2313b0 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 -- 172.21.15.112:0/3497111565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 msgr2=0x560a4b5f1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b5f1a20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009fd0 tx=0x7f66d4014a40 comp rx=0 tx=0).stop 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 -- 172.21.15.112:0/3497111565 shutdown_connections 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b239590 0x560a4b603910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b25a270 0x560a4b603e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 --2- 172.21.15.112:0/3497111565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b5f1a20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 -- 172.21.15.112:0/3497111565 >> 172.21.15.112:0/3497111565 conn(0x560a4b24e3c0 msgr2=0x560a4b6052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 -- 172.21.15.112:0/3497111565 shutdown_connections 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 -- 172.21.15.112:0/3497111565 wait complete. 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.835+0000 7f66ed58b740 1 -- start start 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 Processor -- start 2024-04-25T16:38:18.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b608d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b239590 0x560a4b6092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a4b6128e0 con 0x560a4b2313b0 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a4b612210 con 0x560a4b239590 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a4b612fb0 con 0x560a4b25a270 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66ed58b740 10 client.0 _ll_register_callbacks cb 0x7f66d4009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b608d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b608d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40052/0 (socket says 172.21.15.112:40052) 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 -- 172.21.15.112:0/1051670106 learned_addr learned my addr 172.21.15.112:0/1051670106 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66e4c7d640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b239590 0x560a4b6092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 -- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 msgr2=0x560a4b60c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 -- 172.21.15.112:0/1051670106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b239590 msgr2=0x560a4b6092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b239590 0x560a4b6092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 -- 172.21.15.112:0/1051670106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4b600190 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66dffff640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b608d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005940 tx=0x7f66d4009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee7cabaabbebcd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.836+0000 7f66bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1051670106 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d4004ae0 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1051670106 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f66d4004c80 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d4004730 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1051670106 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f66d4005000 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 10 client.15234 fetch_fsmap learned FSMap version 21 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x560a4b67bcc0 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1051670106 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d4004730 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1051670106 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f66d402d020 con 0x560a4b2313b0 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 10 client.15234 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T16:38:18.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 20 client.15234 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b1ee280 0x560a4b67f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 4 client.15234 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 --> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] -- command(tid 0: {"prefix": "status"}) -- 0x560a4b5681c0 con 0x560a4b1ee280 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.837+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b1ee280 0x560a4b67f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66bf7fe640 10 client.15234 ms_handle_connect on v2:172.21.15.133:6838/99930432 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66df7fe640 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b1ee280 0x560a4b67f3d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e441b211992029d4 server_cookie=def079a57fc64f82 in_seq=0 out_seq=0 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66bf7fe640 1 -- 172.21.15.112:0/1051670106 <== mds.2 v2:172.21.15.133:6838/99930432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x560a4b5681c0 con 0x560a4b1ee280 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66bf7fe640 10 client.15234 handle_command_reply: tid=0 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 client.15234 shutdown 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 msgr2=0x560a4b608d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b608d60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66d4005940 tx=0x7f66d4009ab0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 shutdown_connections 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:6838/99930432,v1:172.21.15.133:6839/99930432] conn(0x560a4b1ee280 0x560a4b67f3d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a4b25a270 0x560a4b60c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a4b239590 0x560a4b6092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 --2- 172.21.15.112:0/1051670106 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a4b2313b0 0x560a4b608d60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66bf7fe640 0 client.15234 ms_handle_reset on v2:172.21.15.133:6838/99930432 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 >> 172.21.15.112:0/1051670106 conn(0x560a4b24e3c0 msgr2=0x560a4b5f66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 shutdown_connections 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 1 -- 172.21.15.112:0/1051670106 wait complete. 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.838+0000 7f66ed58b740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 2, 2024-04-25T16:38:18.843 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24514, 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 108.613819193, 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 79, 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 126.999523333 2024-04-25T16:38:18.844 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f66c80afaf0 msgr2=0x7f66c80b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f66c80afaf0 0x7f66c80b2000 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f66d8004af0 tx=0x7f66d8004a60 comp rx=0 tx=0).stop 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 msgr2=0x7f66e019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 0x7f66e019aa20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f66d000d950 tx=0x7f66d000de10 comp rx=0 tx=0).stop 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 shutdown_connections 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f66c80afaf0 0x7f66c80b2000 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66e0108920 0x7f66e01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66e0103c20 0x7f66e019aa20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 --2- 172.21.15.112:0/1744300418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e01029e0 0x7f66e019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 >> 172.21.15.112:0/1744300418 conn(0x7f66e00fdbd0 msgr2=0x7f66e0109d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 shutdown_connections 2024-04-25T16:38:18.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.839+0000 7f66e66e0640 1 -- 172.21.15.112:0/1744300418 wait complete. 2024-04-25T16:38:18.849 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 -- 172.21.15.112:0/905779756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e841029e0 msgr2=0x7f8e841036e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 --2- 172.21.15.112:0/905779756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e841029e0 0x7f8e841036e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009900 tx=0x7f8e7c02a410 comp rx=0 tx=0).stop 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 -- 172.21.15.112:0/905779756 shutdown_connections 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 --2- 172.21.15.112:0/905779756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e84103c20 0x7f8e84104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 --2- 172.21.15.112:0/905779756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e84108920 0x7f8e8410ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 --2- 172.21.15.112:0/905779756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e841029e0 0x7f8e841036e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 -- 172.21.15.112:0/905779756 >> 172.21.15.112:0/905779756 conn(0x7f8e840fdbd0 msgr2=0x7f8e84100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 -- 172.21.15.112:0/905779756 shutdown_connections 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.959+0000 7f8e88d84640 1 -- 172.21.15.112:0/905779756 wait complete. 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 Processor -- start 2024-04-25T16:38:18.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 -- start start 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e841029e0 0x7f8e84199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e84103c20 0x7f8e84199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 0x7f8e841a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8410e4e0 con 0x7f8e84108920 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8410e360 con 0x7f8e84103c20 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e88d84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8410e660 con 0x7f8e841029e0 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 0x7f8e841a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 0x7f8e841a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40058/0 (socket says 172.21.15.112:40058) 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 -- 172.21.15.112:0/3895675701 learned_addr learned my addr 172.21.15.112:0/3895675701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e8259c640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e841029e0 0x7f8e84199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e84103c20 0x7f8e84199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 -- 172.21.15.112:0/3895675701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e841029e0 msgr2=0x7f8e84199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e841029e0 0x7f8e84199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 -- 172.21.15.112:0/3895675701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e84103c20 msgr2=0x7f8e84199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e84103c20 0x7f8e84199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 -- 172.21.15.112:0/3895675701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e841a0f80 con 0x7f8e84108920 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.960+0000 7f8e82d9d640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 0x7f8e841a0910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e970 tx=0x7f8e7400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.961+0000 7f8e737fe640 1 -- 172.21.15.112:0/3895675701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400ccc0 con 0x7f8e84108920 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.961+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e841a1270 con 0x7f8e84108920 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.961+0000 7f8e737fe640 1 -- 172.21.15.112:0/3895675701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7400ce60 con 0x7f8e84108920 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.961+0000 7f8e737fe640 1 -- 172.21.15.112:0/3895675701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e74010430 con 0x7f8e84108920 2024-04-25T16:38:18.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.961+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e841a17c0 con 0x7f8e84108920 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e737fe640 1 -- 172.21.15.112:0/3895675701 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f8e74004ff0 con 0x7f8e84108920 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 Processor -- start 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 -- start start 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b044270 0x561a6b3e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b023590 0x561a6b3eea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a6b34a740 con 0x561a6b01b3b0 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a6b2e63a0 con 0x561a6b023590 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.963+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561a6b346700 con 0x561a6b044270 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40066/0 (socket says 172.21.15.112:40066) 2024-04-25T16:38:18.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 -- 172.21.15.112:0/2242342419 learned_addr learned my addr 172.21.15.112:0/2242342419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b044270 0x561a6b3e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b023590 0x561a6b3eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e737fe640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e6c0afaf0 0x7f8e6c0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 -- 172.21.15.112:0/2242342419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b044270 msgr2=0x561a6b3e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b044270 0x561a6b3e7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 -- 172.21.15.112:0/2242342419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b023590 msgr2=0x561a6b3eea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b023590 0x561a6b3eea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 -- 172.21.15.112:0/2242342419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6b302750 con 0x561a6b01b3b0 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b023590 0x561a6b3eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e82d9d640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e6c0afaf0 0x7f8e6c0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e737fe640 1 -- 172.21.15.112:0/3895675701 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f8e740df1e0 con 0x7f8e84108920 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e8259c640 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3e7b00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02a930 tx=0x7f8e7c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3b8380e9815fa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e70ff9640 1 -- 172.21.15.112:0/2242342419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c03f030 con 0x561a6b01b3b0 2024-04-25T16:38:18.970 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e70ff9640 1 -- 172.21.15.112:0/2242342419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7c032ca0 con 0x561a6b01b3b0 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.964+0000 7f8e70ff9640 1 -- 172.21.15.112:0/2242342419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c035070 con 0x561a6b01b3b0 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 -- 172.21.15.112:0/2242342419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 msgr2=0x561a6b3e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3e7b00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c02a930 tx=0x7f8e7c005ed0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e82d9d640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e6c0afaf0 0x7f8e6c0b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e740a7380 tx=0x7f8e740ab040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 -- 172.21.15.112:0/2242342419 shutdown_connections 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b023590 0x561a6b3eea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3e7b00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/2242342419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b044270 0x561a6b3e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 -- 172.21.15.112:0/2242342419 >> 172.21.15.112:0/2242342419 conn(0x561a6b0383c0 msgr2=0x561a6b3e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 -- 172.21.15.112:0/2242342419 shutdown_connections 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.965+0000 7f8e8fc32740 1 -- 172.21.15.112:0/2242342419 wait complete. 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 -- start start 2024-04-25T16:38:18.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 Processor -- start 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b044270 0x561a6b3f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3ef3f0 con 0x561a6b01b3b0 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a6b2d1fe0 con 0x561a6b044270 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3ef570 con 0x561a6b023590 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8fc32740 10 client.0 _ll_register_callbacks cb 0x561a6b2e8150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40082/0 (socket says 172.21.15.112:40082) 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 -- 172.21.15.112:0/3582918956 learned_addr learned my addr 172.21.15.112:0/3582918956 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e8259c640 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 -- 172.21.15.112:0/3582918956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 msgr2=0x561a6b3f1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 -- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b044270 msgr2=0x561a6b3f88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b044270 0x561a6b3f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.966+0000 7f8e81d9b640 1 -- 172.21.15.112:0/3582918956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6b05c2d0 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f1420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e78014ca0 tx=0x7f8e78011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28813b4c56b3b4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e69ffb640 1 -- 172.21.15.112:0/3582918956 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78014270 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e69ffb640 1 -- 172.21.15.112:0/3582918956 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f8e78004b20 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e7c032ca0 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e69ffb640 1 -- 172.21.15.112:0/3582918956 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e78014e50 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 10 client.15246 fetch_fsmap learned FSMap version 21 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561a6b1266b0 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e69ffb640 1 -- 172.21.15.112:0/3582918956 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7c032ca0 con 0x561a6b01b3b0 2024-04-25T16:38:18.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e69ffb640 1 -- 172.21.15.112:0/3582918956 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e78012450 con 0x561a6b01b3b0 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 10 client.15246 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 10 client.15246 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 20 client.15246 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b002e40 0x561a6b3ff150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 4 client.15246 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.967+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 --> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a6b358350 con 0x561a6b002e40 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.968+0000 7f8e8259c640 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b002e40 0x561a6b3ff150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.968+0000 7f8e69ffb640 10 client.15246 ms_handle_connect on v2:172.21.15.116:6836/797562383 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.968+0000 7f8e8259c640 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b002e40 0x561a6b3ff150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4f9ffaede1d8e7a server_cookie=164626c0d61c4f4a in_seq=0 out_seq=0 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e69ffb640 1 -- 172.21.15.112:0/3582918956 <== mds.1 v2:172.21.15.116:6836/797562383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561a6b358350 con 0x561a6b002e40 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e69ffb640 10 client.15246 handle_command_reply: tid=0 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 client.15246 shutdown 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 msgr2=0x561a6b3f1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f1420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e78014ca0 tx=0x7f8e78011810 comp rx=0 tx=0).stop 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 shutdown_connections 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b002e40 0x561a6b3ff150 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e69ffb640 0 client.15246 ms_handle_reset on v2:172.21.15.116:6836/797562383 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b044270 0x561a6b3f88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/3582918956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f1420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 >> 172.21.15.112:0/3582918956 conn(0x561a6b0383c0 msgr2=0x561a6b3dfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 shutdown_connections 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 1 -- 172.21.15.112:0/3582918956 wait complete. 2024-04-25T16:38:18.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.969+0000 7f8e8fc32740 20 client.15246 trim_cache size 0 max 16384 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.990+0000 7f8e8fc32740 1 Processor -- start 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.990+0000 7f8e8fc32740 1 -- start start 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 0x561a6b3dba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b01b3b0 0x561a6b3f8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3fd490 con 0x561a6b002e40 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3fdb60 con 0x561a6b01b3b0 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3fe230 con 0x561a6b023590 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 0x561a6b3dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 0x561a6b3dba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40086/0 (socket says 172.21.15.112:40086) 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 -- 172.21.15.112:0/267685542 learned_addr learned my addr 172.21.15.112:0/267685542 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b01b3b0 0x561a6b3f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 -- 172.21.15.112:0/267685542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 msgr2=0x561a6b3f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 -- 172.21.15.112:0/267685542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b01b3b0 msgr2=0x561a6b3f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b01b3b0 0x561a6b3f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 -- 172.21.15.112:0/267685542 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a6b3ec300 con 0x561a6b002e40 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 0x561a6b3dba20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8e78002230 tx=0x7f8e780147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2256688025c659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e6bfff640 1 -- 172.21.15.112:0/267685542 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e78030070 con 0x561a6b002e40 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e6bfff640 1 -- 172.21.15.112:0/267685542 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e78014e50 con 0x561a6b002e40 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.991+0000 7f8e6bfff640 1 -- 172.21.15.112:0/267685542 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7802b6a0 con 0x561a6b002e40 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 -- 172.21.15.112:0/267685542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 msgr2=0x561a6b3dba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 0x561a6b3dba20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8e78002230 tx=0x7f8e780147e0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 -- 172.21.15.112:0/267685542 shutdown_connections 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b01b3b0 0x561a6b3f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/267685542 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b002e40 0x561a6b3dba20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 -- 172.21.15.112:0/267685542 >> 172.21.15.112:0/267685542 conn(0x561a6b0383c0 msgr2=0x561a6b3efa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 -- 172.21.15.112:0/267685542 shutdown_connections 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 -- 172.21.15.112:0/267685542 wait complete. 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.992+0000 7f8e8fc32740 1 -- start start 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 Processor -- start 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b002e40 0x561a6b3f4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3fefd0 con 0x561a6b01b3b0 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3fe900 con 0x561a6b002e40 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561a6b3ff6a0 con 0x561a6b023590 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 10 client.0 _ll_register_callbacks cb 0x7f8e780148b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:18.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40102/0 (socket says 172.21.15.112:40102) 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 -- 172.21.15.112:0/879210304 learned_addr learned my addr 172.21.15.112:0/879210304 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b002e40 0x561a6b3f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 -- 172.21.15.112:0/879210304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 msgr2=0x561a6b3f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 -- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b002e40 msgr2=0x561a6b3f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b002e40 0x561a6b3f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 -- 172.21.15.112:0/879210304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6b3f8380 con 0x561a6b01b3b0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8259c640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f4e80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009850 tx=0x7f8e7c0315c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d90939cc754c4d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e69ffb640 1 -- 172.21.15.112:0/879210304 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c03f040 con 0x561a6b01b3b0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e69ffb640 1 -- 172.21.15.112:0/879210304 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f8e7c032c20 con 0x561a6b01b3b0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.993+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e78014e50 con 0x561a6b01b3b0 2024-04-25T16:38:18.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e69ffb640 1 -- 172.21.15.112:0/879210304 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8e7c05b050 con 0x561a6b01b3b0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 10 client.15252 fetch_fsmap learned FSMap version 21 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x561a6b468030 con 0x561a6b01b3b0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e69ffb640 1 -- 172.21.15.112:0/879210304 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e78014e50 con 0x561a6b01b3b0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e69ffb640 1 -- 172.21.15.112:0/879210304 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7f8e7c0053f0 con 0x561a6b01b3b0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 10 client.15252 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 10 client.15252 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 20 client.15252 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b044270 0x561a6b46b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 4 client.15252 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 --> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] -- command(tid 0: {"prefix": "status"}) -- 0x561a6b3521c0 con 0x561a6b044270 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.994+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b044270 0x561a6b46b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e69ffb640 10 client.15252 ms_handle_connect on v2:172.21.15.116:6836/797562383 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e81d9b640 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b044270 0x561a6b46b4b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e524af66825fe44f server_cookie=98a7ef719014b370 in_seq=0 out_seq=0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e69ffb640 1 -- 172.21.15.112:0/879210304 <== mds.1 v2:172.21.15.116:6836/797562383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x561a6b3521c0 con 0x561a6b044270 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e69ffb640 10 client.15252 handle_command_reply: tid=0 2024-04-25T16:38:18.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 client.15252 shutdown 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 msgr2=0x561a6b3f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f4e80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c009850 tx=0x7f8e7c0315c0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 shutdown_connections 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:6836/797562383,v1:172.21.15.116:6837/797562383] conn(0x561a6b044270 0x561a6b46b4b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561a6b023590 0x561a6b3f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561a6b01b3b0 0x561a6b3f4e80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 --2- 172.21.15.112:0/879210304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a6b002e40 0x561a6b3f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e69ffb640 0 client.15252 ms_handle_reset on v2:172.21.15.116:6836/797562383 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 >> 172.21.15.112:0/879210304 conn(0x561a6b0383c0 msgr2=0x561a6b3e06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 shutdown_connections 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 1 -- 172.21.15.112:0/879210304 wait complete. 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.995+0000 7f8e8fc32740 20 client.15252 trim_cache size 0 max 16384 2024-04-25T16:38:19.000 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 1, 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24524, 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 109.776557451, 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 79, 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 129.91102787 2024-04-25T16:38:19.001 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e6c0afaf0 msgr2=0x7f8e6c0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e6c0afaf0 0x7f8e6c0b2000 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e740a7380 tx=0x7f8e740ab040 comp rx=0 tx=0).stop 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 msgr2=0x7f8e841a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 0x7f8e841a0910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400e970 tx=0x7f8e7400ed70 comp rx=0 tx=0).stop 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 shutdown_connections 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8e84108920 0x7f8e841a0910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e84103c20 0x7f8e84199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f8e6c0afaf0 0x7f8e6c0b2000 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 --2- 172.21.15.112:0/3895675701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e841029e0 0x7f8e84199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 >> 172.21.15.112:0/3895675701 conn(0x7f8e840fdbd0 msgr2=0x7f8e84104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 shutdown_connections 2024-04-25T16:38:19.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:18.996+0000 7f8e88d84640 1 -- 172.21.15.112:0/3895675701 wait complete. 2024-04-25T16:38:19.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 -- 172.21.15.112:0/3280558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 msgr2=0x7fa1c41054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 --2- 172.21.15.112:0/3280558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41054b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009900 tx=0x7fa1b802a410 comp rx=0 tx=0).stop 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 -- 172.21.15.112:0/3280558445 shutdown_connections 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 --2- 172.21.15.112:0/3280558445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4102250 0x7fa1c410bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 --2- 172.21.15.112:0/3280558445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1c4101750 0x7fa1c4101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 --2- 172.21.15.112:0/3280558445 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41054b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 -- 172.21.15.112:0/3280558445 >> 172.21.15.112:0/3280558445 conn(0x7fa1c40fd350 msgr2=0x7fa1c40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 -- 172.21.15.112:0/3280558445 shutdown_connections 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.115+0000 7fa1cb196640 1 -- 172.21.15.112:0/3280558445 wait complete. 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 Processor -- start 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 -- start start 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1c4101750 0x7fa1c419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4102250 0x7fa1c419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410f610 con 0x7fa1c41050a0 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410f490 con 0x7fa1c4101750 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1cb196640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410f790 con 0x7fa1c4102250 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40116/0 (socket says 172.21.15.112:40116) 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 -- 172.21.15.112:0/2701124194 learned_addr learned my addr 172.21.15.112:0/2701124194 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1c4101750 0x7fa1c419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 -- 172.21.15.112:0/2701124194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4102250 msgr2=0x7fa1c419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4102250 0x7fa1c419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 -- 172.21.15.112:0/2701124194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1c4101750 msgr2=0x7fa1c419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1c4101750 0x7fa1c419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.116+0000 7fa1c9733640 1 -- 172.21.15.112:0/2701124194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c41a1db0 con 0x7fa1c41050a0 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.117+0000 7fa1c9733640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41a1740 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00cad0 tx=0x7fa1bc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.117+0000 7fa1c1ffb640 1 -- 172.21.15.112:0/2701124194 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00ae20 con 0x7fa1c41050a0 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.117+0000 7fa1c1ffb640 1 -- 172.21.15.112:0/2701124194 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1bc005020 con 0x7fa1c41050a0 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.117+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c41a20a0 con 0x7fa1c41050a0 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.117+0000 7fa1c1ffb640 1 -- 172.21.15.112:0/2701124194 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc00f6b0 con 0x7fa1c41050a0 2024-04-25T16:38:19.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.117+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41a25a0 con 0x7fa1c41050a0 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c1ffb640 1 -- 172.21.15.112:0/2701124194 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7fa1bc0051d0 con 0x7fa1c41050a0 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 Processor -- start 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 -- start start 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858ad9f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ad9f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55858ad04740 con 0x55858a9d53b0 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55858aca03a0 con 0x55858a9dd590 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1d204a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55858ad00700 con 0x55858a9fe270 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c1ffb640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1b40afaf0 0x7fa1b40b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c8f32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ad9f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c8f32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ad9f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40124/0 (socket says 172.21.15.112:40124) 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858ad9f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c8f32640 1 -- 172.21.15.112:0/4229767208 learned_addr learned my addr 172.21.15.112:0/4229767208 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.120+0000 7fa1c9733640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1b40afaf0 0x7fa1b40b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c1ffb640 1 -- 172.21.15.112:0/2701124194 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fa1bc0decc0 con 0x7fa1c41050a0 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c8f32640 1 -- 172.21.15.112:0/4229767208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 msgr2=0x55858ad9f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858ad9f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c8f32640 1 -- 172.21.15.112:0/4229767208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 msgr2=0x55858ada67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c8f32640 1 -- 172.21.15.112:0/4229767208 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858acbc750 con 0x55858a9d53b0 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ad9f8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009230 tx=0x7fa1b8031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7912a408095546bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1c9733640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1b40afaf0 0x7fa1b40b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0a6e60 tx=0x7fa1bc0d58b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1aeffd640 1 -- 172.21.15.112:0/4229767208 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8035030 con 0x55858a9d53b0 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1aeffd640 1 -- 172.21.15.112:0/4229767208 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8040070 con 0x55858a9d53b0 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.121+0000 7fa1aeffd640 1 -- 172.21.15.112:0/4229767208 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8003ad0 con 0x55858a9d53b0 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 -- 172.21.15.112:0/4229767208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 msgr2=0x55858ad9f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ad9f8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009230 tx=0x7fa1b8031c50 comp rx=0 tx=0).stop 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 -- 172.21.15.112:0/4229767208 shutdown_connections 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ad9f8f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 --2- 172.21.15.112:0/4229767208 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858ad9f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 -- 172.21.15.112:0/4229767208 >> 172.21.15.112:0/4229767208 conn(0x55858a9f23c0 msgr2=0x55858ad9ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 -- 172.21.15.112:0/4229767208 shutdown_connections 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 -- 172.21.15.112:0/4229767208 wait complete. 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 -- start start 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.122+0000 7fa1d204a740 1 Processor -- start 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ada9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858adb06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55858ada71e0 con 0x55858a9d53b0 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55858ac8bfe0 con 0x55858a9dd590 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55858ada7360 con 0x55858a9fe270 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1d204a740 10 client.0 _ll_register_callbacks cb 0x55858aca2150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ada9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ada9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40128/0 (socket says 172.21.15.112:40128) 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 -- 172.21.15.112:0/2234684907 learned_addr learned my addr 172.21.15.112:0/2234684907 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:19.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858adb06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 -- 172.21.15.112:0/2234684907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 msgr2=0x55858adb06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858adb06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 -- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 msgr2=0x55858ada9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 -- 172.21.15.112:0/2234684907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858aa162d0 con 0x55858a9d53b0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ada9220 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0013af0 tx=0x7fa1b000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e111633c74d21c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.123+0000 7fa1a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1a3fff640 1 -- 172.21.15.112:0/2234684907 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0013320 con 0x55858a9d53b0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1a3fff640 1 -- 172.21.15.112:0/2234684907 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fa1b0011a60 con 0x55858a9d53b0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b8040070 con 0x55858a9d53b0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1a3fff640 1 -- 172.21.15.112:0/2234684907 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa1b0026050 con 0x55858a9d53b0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 10 client.15264 fetch_fsmap learned FSMap version 21 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55858aae06b0 con 0x55858a9d53b0 2024-04-25T16:38:19.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1a3fff640 1 -- 172.21.15.112:0/2234684907 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8040070 con 0x55858a9d53b0 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1a3fff640 1 -- 172.21.15.112:0/2234684907 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa1b001e3c0 con 0x55858a9d53b0 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 10 client.15264 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 20 client.15264 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9bce40 0x55858adb6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 4 client.15264 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55858ad12350 con 0x55858a9bce40 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.124+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9bce40 0x55858adb6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.125+0000 7fa1a3fff640 10 client.15264 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.125+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9bce40 0x55858adb6f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8ff998025a62b8 server_cookie=18d289b2e952c1db in_seq=0 out_seq=0 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.125+0000 7fa1a3fff640 1 -- 172.21.15.112:0/2234684907 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55858ad12350 con 0x55858a9bce40 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.125+0000 7fa1a3fff640 10 client.15264 handle_command_reply: tid=0 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.125+0000 7fa1d204a740 1 client.15264 shutdown 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 msgr2=0x55858ada9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ada9220 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0013af0 tx=0x7fa1b000da80 comp rx=0 tx=0).stop 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 shutdown_connections 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9bce40 0x55858adb6f60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9fe270 0x55858adb06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ada9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1a3fff640 0 client.15264 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 --2- 172.21.15.112:0/2234684907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ada9220 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 >> 172.21.15.112:0/2234684907 conn(0x55858a9f23c0 msgr2=0x55858ad99d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 shutdown_connections 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 1 -- 172.21.15.112:0/2234684907 wait complete. 2024-04-25T16:38:19.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.126+0000 7fa1d204a740 20 client.15264 trim_cache size 0 max 16384 2024-04-25T16:38:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 Processor -- start 2024-04-25T16:38:19.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 -- start start 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 0x55858ada7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9d53b0 0x55858ada2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9dd590 0x55858adb0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55858adb52a0 con 0x55858a9bce40 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55858adb5970 con 0x55858a9d53b0 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1d204a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55858adb6040 con 0x55858a9dd590 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 0x55858ada7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 0x55858ada7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40136/0 (socket says 172.21.15.112:40136) 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.144+0000 7fa1c3fff640 1 -- 172.21.15.112:0/3536695587 learned_addr learned my addr 172.21.15.112:0/3536695587 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1c3fff640 1 -- 172.21.15.112:0/3536695587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9dd590 msgr2=0x55858adb0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9dd590 0x55858adb0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1c3fff640 1 -- 172.21.15.112:0/3536695587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9d53b0 msgr2=0x55858ada2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9d53b0 0x55858ada2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1c3fff640 1 -- 172.21.15.112:0/3536695587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55858ada40c0 con 0x55858a9bce40 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 0x55858ada7e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00135b0 tx=0x7fa1b00135e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ec2d61d37f9ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1ae7fc640 1 -- 172.21.15.112:0/3536695587 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b001ee30 con 0x55858a9bce40 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1ae7fc640 1 -- 172.21.15.112:0/3536695587 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b0013320 con 0x55858a9bce40 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.145+0000 7fa1ae7fc640 1 -- 172.21.15.112:0/3536695587 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0004030 con 0x55858a9bce40 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- 172.21.15.112:0/3536695587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 msgr2=0x55858ada7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 0x55858ada7e70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00135b0 tx=0x7fa1b00135e0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- 172.21.15.112:0/3536695587 shutdown_connections 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9dd590 0x55858adb0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9d53b0 0x55858ada2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3536695587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9bce40 0x55858ada7e70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- 172.21.15.112:0/3536695587 >> 172.21.15.112:0/3536695587 conn(0x55858a9f23c0 msgr2=0x55858ada7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- 172.21.15.112:0/3536695587 shutdown_connections 2024-04-25T16:38:19.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- 172.21.15.112:0/3536695587 wait complete. 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- start start 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 Processor -- start 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9bce40 0x55858ae1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ae20320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ae22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.146+0000 7fa1d204a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55858adb6de0 con 0x55858a9d53b0 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1d204a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55858adb6710 con 0x55858a9dd590 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1d204a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55858adb74b0 con 0x55858a9bce40 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1d204a740 10 client.0 _ll_register_callbacks cb 0x7fa1b0013730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ae20320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ae20320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40150/0 (socket says 172.21.15.112:40150) 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 -- 172.21.15.112:0/3217930974 learned_addr learned my addr 172.21.15.112:0/3217930974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9bce40 0x55858ae1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ae22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 -- 172.21.15.112:0/3217930974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9bce40 msgr2=0x55858ae1ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9bce40 0x55858ae1ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 -- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 msgr2=0x55858ae22960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ae22960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 -- 172.21.15.112:0/3217930974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55858ae230c0 con 0x55858a9d53b0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9bce40 0x55858ae1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1c8f32640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ae20320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8005f70 tx=0x7fa1b8009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609e7dea0840cefe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1a3fff640 1 -- 172.21.15.112:0/3217930974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8035040 con 0x55858a9d53b0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1a3fff640 1 -- 172.21.15.112:0/3217930974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fa1b8045670 con 0x55858a9d53b0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1b0013320 con 0x55858a9d53b0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1a3fff640 1 -- 172.21.15.112:0/3217930974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa1b8045870 con 0x55858a9d53b0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1d204a740 10 client.15270 fetch_fsmap learned FSMap version 21 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.147+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55858ae24340 con 0x55858a9d53b0 2024-04-25T16:38:19.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1a3fff640 1 -- 172.21.15.112:0/3217930974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b0013320 con 0x55858a9d53b0 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1a3fff640 1 -- 172.21.15.112:0/3217930974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 21) ==== 8338+0+0 (secure 0 0 0) 0x7fa1b8052a90 con 0x55858a9d53b0 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1d204a740 10 client.15270 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1d204a740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1d204a740 20 client.15270 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9fe270 0x55858ae255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1d204a740 4 client.15270 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "status"}) -- 0x55858ad0c1c0 con 0x55858a9fe270 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.148+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9fe270 0x55858ae255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1a3fff640 10 client.15270 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1c3fff640 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9fe270 0x55858ae255a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d5c96efde1bbe4 server_cookie=d10d58ed4260a414 in_seq=0 out_seq=0 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1a3fff640 1 -- 172.21.15.112:0/3217930974 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55858ad0c1c0 con 0x55858a9fe270 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1a3fff640 10 client.15270 handle_command_reply: tid=0 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 client.15270 shutdown 2024-04-25T16:38:19.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 msgr2=0x55858ae20320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ae20320 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8005f70 tx=0x7fa1b8009830 comp rx=0 tx=0).stop 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 shutdown_connections 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55858a9dd590 0x55858ae22960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55858a9d53b0 0x55858ae20320 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55858a9fe270 0x55858ae255a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 --2- 172.21.15.112:0/3217930974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55858a9bce40 0x55858ae1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1a3fff640 0 client.15270 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 >> 172.21.15.112:0/3217930974 conn(0x55858a9f23c0 msgr2=0x55858ada2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 shutdown_connections 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 1 -- 172.21.15.112:0/3217930974 wait complete. 2024-04-25T16:38:19.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.149+0000 7fa1d204a740 20 client.15270 trim_cache size 0 max 16384 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "a7905982-0320-11ef-bc93-c7b262605968", 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 24560, 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 111.955184308, 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 21, 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 79, 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 75, 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 118.906661809 2024-04-25T16:38:19.155 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:38:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1b40afaf0 msgr2=0x7fa1b40b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1b40afaf0 0x7fa1b40b2000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0a6e60 tx=0x7fa1bc0d58b0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 msgr2=0x7fa1c41a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41a1740 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00cad0 tx=0x7fa1bc00cf90 comp rx=0 tx=0).stop 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 shutdown_connections 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fa1b40afaf0 0x7fa1b40b2000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1c4102250 0x7fa1c419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1c41050a0 0x7fa1c41a1740 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 --2- 172.21.15.112:0/2701124194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1c4101750 0x7fa1c419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 >> 172.21.15.112:0/2701124194 conn(0x7fa1c40fd350 msgr2=0x7fa1c4105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 shutdown_connections 2024-04-25T16:38:19.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.150+0000 7fa1cb196640 1 -- 172.21.15.112:0/2701124194 wait complete. 2024-04-25T16:38:19.159 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T16:38:19.169 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T16:38:19.169 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T16:38:19.169 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T16:38:19.169 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T16:38:19.170 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T16:38:19.170 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:38:19.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 -- 172.21.15.112:0/1211306672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240108970 msgr2=0x7f924010ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 --2- 172.21.15.112:0/1211306672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240108970 0x7f924010ad90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9230009900 tx=0x7f923002a410 comp rx=0 tx=0).stop 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 -- 172.21.15.112:0/1211306672 shutdown_connections 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 --2- 172.21.15.112:0/1211306672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240103c70 0x7f9240104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 --2- 172.21.15.112:0/1211306672 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240108970 0x7f924010ad90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 --2- 172.21.15.112:0/1211306672 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9240102a30 0x7f9240103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 -- 172.21.15.112:0/1211306672 >> 172.21.15.112:0/1211306672 conn(0x7f92400fdc20 msgr2=0x7f92401000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 -- 172.21.15.112:0/1211306672 shutdown_connections 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.280+0000 7f924764a640 1 -- 172.21.15.112:0/1211306672 wait complete. 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 Processor -- start 2024-04-25T16:38:19.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 -- start start 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 0x7f92400758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240103c70 0x7f924019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9240108970 0x7f92401a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f924010e560 con 0x7f9240102a30 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f924010e3e0 con 0x7f9240108970 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f924764a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f924010e6e0 con 0x7f9240103c70 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f92453e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 0x7f92400758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f92453e6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 0x7f92400758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40176/0 (socket says 172.21.15.112:40176) 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f92453e6640 1 -- 172.21.15.112:0/1807380798 learned_addr learned my addr 172.21.15.112:0/1807380798 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f9245be7640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9240108970 0x7f92401a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.281+0000 7f9244be5640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240103c70 0x7f924019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f92453e6640 1 -- 172.21.15.112:0/1807380798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240103c70 msgr2=0x7f924019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f92453e6640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240103c70 0x7f924019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f92453e6640 1 -- 172.21.15.112:0/1807380798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9240108970 msgr2=0x7f92401a1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f92453e6640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9240108970 0x7f92401a1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f92453e6640 1 -- 172.21.15.112:0/1807380798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92401a1d20 con 0x7f9240102a30 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f92453e6640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 0x7f92400758a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f923800eb90 tx=0x7f923800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238003b70 con 0x7f9240102a30 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f923800ee90 con 0x7f9240102a30 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92401a1fb0 con 0x7f9240102a30 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92380197b0 con 0x7f9240102a30 2024-04-25T16:38:19.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.282+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92401a2470 con 0x7f9240102a30 2024-04-25T16:38:19.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.283+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9240107780 con 0x7f9240102a30 2024-04-25T16:38:19.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.284+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 18) ==== 147521+0+0 (secure 0 0 0) 0x7f9238004030 con 0x7f9240102a30 2024-04-25T16:38:19.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.285+0000 7f923e7fc640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f92280afae0 0x7f92280b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:19.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.285+0000 7f9244be5640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f92280afae0 0x7f92280b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:19.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.285+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f92380df570 con 0x7f9240102a30 2024-04-25T16:38:19.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.285+0000 7f9244be5640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f92280afae0 0x7f92280b1ff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9240075240 tx=0x7f9234005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:19.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.289+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f92380a8110 con 0x7f9240102a30 2024-04-25T16:38:19.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:19 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3723527923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:38:19.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:19 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2682341072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:19.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:19 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3723527923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:38:19.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:19 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2682341072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:19.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:19.517+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f924002db50 con 0x7f9240102a30 2024-04-25T16:38:19.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:19 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3723527923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:38:19.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:19 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2682341072' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:38:20.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.232+0000 7f923e7fc640 1 -- 172.21.15.112:0/1807380798 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f92380b0190 con 0x7f9240102a30 2024-04-25T16:38:20.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f92280afae0 msgr2=0x7f92280b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:20.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f92280afae0 0x7f92280b1ff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9240075240 tx=0x7f9234005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:20.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 msgr2=0x7f92400758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:20.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 0x7f92400758a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f923800eb90 tx=0x7f923800c6a0 comp rx=0 tx=0).stop 2024-04-25T16:38:20.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 shutdown_connections 2024-04-25T16:38:20.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f92280afae0 0x7f92280b1ff0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:20.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9240103c70 0x7f924019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:20.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9240108970 0x7f92401a1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:20.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 --2- 172.21.15.112:0/1807380798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9240102a30 0x7f92400758a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:20.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 >> 172.21.15.112:0/1807380798 conn(0x7f92400fdc20 msgr2=0x7f9240104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:20.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 shutdown_connections 2024-04-25T16:38:20.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:20.235+0000 7f924764a640 1 -- 172.21.15.112:0/1807380798 wait complete. 2024-04-25T16:38:20.254 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T16:38:20.282 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T16:38:20.282 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T16:38:20.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:20.283 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T16:38:20.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:20.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T16:38:20.365 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T16:38:20.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:38:20.366 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T16:38:20.367 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T16:38:20.367 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:20.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T16:38:20.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-mon[32712]: pgmap v341: 97 pgs: 1 activating, 96 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T16:38:20.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1807380798' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T16:38:20.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: ignoring --setuser ceph since I am not root 2024-04-25T16:38:20.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: ignoring --setgroup ceph since I am not root 2024-04-25T16:38:20.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-mgr[34099]: -- 172.21.15.116:0/3836653026 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563dd63305a0 con 0x563ddb0ba000 2024-04-25T16:38:20.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:20.441+0000 7f5e37d15200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:38:20.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-mon[29122]: pgmap v341: 97 pgs: 1 activating, 96 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T16:38:20.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1807380798' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T16:38:20.595 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: ignoring --setuser ceph since I am not root 2024-04-25T16:38:20.595 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: ignoring --setgroup ceph since I am not root 2024-04-25T16:38:20.596 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:20.449+0000 7fb5c6bb8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:38:20.626 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-mon[32691]: pgmap v341: 97 pgs: 1 activating, 96 active+clean; 641 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T16:38:20.627 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1807380798' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T16:38:20.627 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: ignoring --setuser ceph since I am not root 2024-04-25T16:38:20.627 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: ignoring --setgroup ceph since I am not root 2024-04-25T16:38:20.627 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-mgr[34501]: -- 172.21.15.133:0/3585490654 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ebe9b25a0 con 0x561ec373c000 2024-04-25T16:38:20.627 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:20.440+0000 7fb862a83200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T16:38:20.627 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:20.628+0000 7fb862a83200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:38:20.648 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:38:20.914 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:20.651+0000 7fb5c6bb8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:38:20.915 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:20 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:20.762+0000 7fb5c6bb8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:38:20.949 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:20.732+0000 7fb862a83200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:38:21.008 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:20.628+0000 7f5e37d15200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:38:21.008 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:20.731+0000 7f5e37d15200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T16:38:21.170 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:21.083+0000 7fb5c6bb8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:38:21.170 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:21 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:21.172+0000 7fb5c6bb8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:38:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:21 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/1807380798' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T16:38:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:21 smithi133 ceph-mon[32691]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T16:38:21.449 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:21.045+0000 7fb862a83200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:38:21.450 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:21.132+0000 7fb862a83200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:38:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:21 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/1807380798' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T16:38:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:21 smithi116 ceph-mon[32712]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T16:38:21.508 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:21.042+0000 7f5e37d15200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T16:38:21.509 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:21.129+0000 7f5e37d15200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T16:38:21.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:21 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/1807380798' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T16:38:21.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:21 smithi112 ceph-mon[29122]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 -- 172.21.15.112:0/1952745869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102f80 msgr2=0x7fea10111010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 --2- 172.21.15.112:0/1952745869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102f80 0x7fea10111010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b0a0 tx=0x7fea0800b3b0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 -- 172.21.15.112:0/1952745869 shutdown_connections 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 --2- 172.21.15.112:0/1952745869 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102f80 0x7fea10111010 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 --2- 172.21.15.112:0/1952745869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea101091a0 0x7fea101095b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 --2- 172.21.15.112:0/1952745869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102630 0x7fea10102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 -- 172.21.15.112:0/1952745869 >> 172.21.15.112:0/1952745869 conn(0x7fea100fe3f0 msgr2=0x7fea10100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:22.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 -- 172.21.15.112:0/1952745869 shutdown_connections 2024-04-25T16:38:22.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.459+0000 7fea14ddc700 1 -- 172.21.15.112:0/1952745869 wait complete. 2024-04-25T16:38:22.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.460+0000 7fea14ddc700 1 Processor -- start 2024-04-25T16:38:22.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.460+0000 7fea14ddc700 1 -- start start 2024-04-25T16:38:22.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.460+0000 7fea14ddc700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 0x7fea1019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.460+0000 7fea14ddc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102f80 0x7fea1019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea14ddc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea101091a0 0x7fea1019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea14ddc700 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fea10114440 con 0x7fea10102630 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea14ddc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fea101142c0 con 0x7fea10102f80 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea14ddc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fea101145c0 con 0x7fea101091a0 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 0x7fea1019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 0x7fea1019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40202/0 (socket says 172.21.15.112:40202) 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 -- 172.21.15.112:0/523876881 learned_addr learned my addr 172.21.15.112:0/523876881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0dd9b700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102f80 0x7fea1019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0ed9d700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea101091a0 0x7fea1019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 -- 172.21.15.112:0/523876881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea101091a0 msgr2=0x7fea1019f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea101091a0 0x7fea1019f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 -- 172.21.15.112:0/523876881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102f80 msgr2=0x7fea1019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:22.461 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102f80 0x7fea1019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 -- 172.21.15.112:0/523876881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1019fbe0 con 0x7fea10102630 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0dd9b700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102f80 0x7fea1019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea0e59c700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 0x7fea1019ad10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e940 tx=0x7fea0000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00004220 con 0x7fea10102630 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea000043c0 con 0x7fea10102630 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00003c60 con 0x7fea10102630 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.461+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1019fed0 con 0x7fea10102630 2024-04-25T16:38:22.462 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.462+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea101a77f0 con 0x7fea10102630 2024-04-25T16:38:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.463+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea10051190 con 0x7fea10102630 2024-04-25T16:38:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.464+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 147538+0+0 (secure 0 0 0) 0x7fea0000c7e0 con 0x7fea10102630 2024-04-25T16:38:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.465+0000 7fe9ff7fe700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 0x7fe9f80b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.465+0000 7fea0dd9b700 1 -- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 msgr2=0x7fe9f80b20e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.465+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fea000def90 con 0x7fea10102630 2024-04-25T16:38:22.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.465+0000 7fea0dd9b700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 0x7fe9f80b20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:38:22.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.469+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea000a7b60 con 0x7fea10102630 2024-04-25T16:38:22.595 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:22 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:22.399+0000 7fb5c6bb8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:38:22.595 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:22 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:22.586+0000 7fb5c6bb8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:38:22.655 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:22.312+0000 7f5e37d15200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:38:22.655 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:22.485+0000 7f5e37d15200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:38:22.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.666+0000 7fea0dd9b700 1 -- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 msgr2=0x7fe9f80b20e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:22.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.666+0000 7fea0dd9b700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 0x7fe9f80b20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:38:22.680 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:22.332+0000 7fb862a83200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T16:38:22.680 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:22.508+0000 7fb862a83200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T16:38:22.680 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:22.682+0000 7fb862a83200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:38:22.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.723+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea1019c070 con 0x7fea10102630 2024-04-25T16:38:22.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.723+0000 7fe9ff7fe700 1 -- 172.21.15.112:0/523876881 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+73 (secure 0 0 0) 0x7fea000a7460 con 0x7fea10102630 2024-04-25T16:38:22.722 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 msgr2=0x7fe9f80b20e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 0x7fe9f80b20e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 msgr2=0x7fea1019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 0x7fea1019ad10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fea0000e940 tx=0x7fea0000ec50 comp rx=0 tx=0).stop 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 shutdown_connections 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea101091a0 0x7fea1019f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fea10102630 0x7fea1019ad10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fe9f80afbd0 0x7fe9f80b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.725+0000 7fea14ddc700 1 --2- 172.21.15.112:0/523876881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea10102f80 0x7fea1019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.726+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 >> 172.21.15.112:0/523876881 conn(0x7fea100fe3f0 msgr2=0x7fea1010a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.726+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 shutdown_connections 2024-04-25T16:38:22.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:22.726+0000 7fea14ddc700 1 -- 172.21.15.112:0/523876881 wait complete. 2024-04-25T16:38:22.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:22 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/523876881' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T16:38:22.949 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:22 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/523876881' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T16:38:22.950 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:22.656+0000 7f5e37d15200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:38:23.040 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:22 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/523876881' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T16:38:23.041 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:22 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:22.767+0000 7fb5c6bb8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T16:38:23.246 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:22.951+0000 7f5e37d15200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:38:23.246 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:23.044+0000 7f5e37d15200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:38:23.277 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:22.978+0000 7fb862a83200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:38:23.278 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:23.072+0000 7fb862a83200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:38:23.345 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:23.071+0000 7fb5c6bb8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T16:38:23.345 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:23.167+0000 7fb5c6bb8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T16:38:23.476 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:38:23.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:23.477 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T16:38:23.492 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T16:38:23.492 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T16:38:23.493 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:38:23.494 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T16:38:23.494 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T16:38:23.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T16:38:23.494 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42124sec preferred_lft 42124sec 2024-04-25T16:38:23.494 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::7262:a08e:a16d:21d7/64 scope link noprefixroute 2024-04-25T16:38:23.494 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T16:38:23.494 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T16:38:23.494 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:38:23.495 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T16:38:23.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-25T16:38:23.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-25T16:38:23.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-25T16:38:23.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T16:38:23.495 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T16:38:23.508 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:23.248+0000 7f5e37d15200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:38:23.508 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:23.445+0000 7f5e37d15200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:38:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:38:23.637 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:23.373+0000 7fb5c6bb8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:38:23.638 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:23 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:23.592+0000 7fb5c6bb8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:38:23.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:23.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:23.683 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T16:38:23.699 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:23.279+0000 7fb862a83200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T16:38:23.699 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:23.477+0000 7fb862a83200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T16:38:23.710 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T16:38:23.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:23.712 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T16:38:23.785 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T16:38:23.785 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T16:38:23.786 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T16:38:23.786 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T16:38:23.786 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T16:38:23.786 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:38:23.786 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T16:38:23.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T16:38:23.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T16:38:23.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T16:38:23.787 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T16:38:23.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:38:24.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:24.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:24.039 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T16:38:24.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:24.053 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-25T16:38:24.109 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:38:24.109 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T16:38:24.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:38:24.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T16:38:24.109 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T16:38:24.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:38:24.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:24.228 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T16:38:24.228 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:38:24.228 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T16:38:24.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T16:38:24.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T16:38:24.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T16:38:24.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T16:38:24.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T16:38:24.229 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T16:38:24.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:38:24.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:24.357 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T16:38:24.357 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T16:38:24.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-25T16:38:24.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T16:38:24.357 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T16:38:24.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:38:24.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:24.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:38:24.437 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:24.449 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:24.084+0000 7fb862a83200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:38:24.451 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T16:38:24.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T16:38:24.451 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:24.475 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:24.044+0000 7f5e37d15200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:38:24.507 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T16:38:24.507 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T16:38:24.508 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/d5283f8f-09b0-4e50-a257-9ae4d1f0c74a 2024-04-25T16:38:24.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:24.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/d5283f8f-09b0-4e50-a257-9ae4d1f0c74a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T16:38:24.568 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:24.256+0000 7fb5c6bb8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T16:38:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T16:38:24.613 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-25T16:38:24.726 INFO:teuthology.orchestra.run.smithi112.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-04-25T16:38:24.727 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-04-25T16:38:24.727 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: resolved to: "172.21.15.112:3300,172.21.15.116:3300,172.21.15.133:3300" 2024-04-25T16:38:24.727 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: trying mount with new device syntax: 0@a7905982-0320-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/d5283f8f-09b0-4e50-a257-9ae4d1f0c74a 2024-04-25T16:38:24.727 INFO:teuthology.orchestra.run.smithi112.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.112:3300/172.21.15.116:3300/172.21.15.133:3300" will pass to kernel 2024-04-25T16:38:24.728 INFO:teuthology.orchestra.run.smithi112.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T16:38:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: You just mounted a file system that supports labels which does not 2024-04-25T16:38:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T16:38:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T16:38:24.728 INFO:teuthology.orchestra.run.smithi112.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T16:38:24.730 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T16:38:24.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:38:24.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:24.758 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:24.476+0000 7f5e37d15200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:38:24.758 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:24.575+0000 7f5e37d15200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:38:24.797 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:38:24.818 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:24.516+0000 7fb862a83200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:38:24.819 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:24.613+0000 7fb862a83200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:38:24.831 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:24.727+0000 7fb5c6bb8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T16:38:24.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:38:24.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:24.888 INFO:teuthology.orchestra.run.smithi112.stdout:client24973 2024-04-25T16:38:24.888 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:38:24.888 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:38:24.905 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:38:24.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:38:24.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:25.004 INFO:teuthology.orchestra.run.smithi112.stdout:client24973 2024-04-25T16:38:25.004 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T16:38:25.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T16:38:25.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:25.060 INFO:teuthology.orchestra.run.smithi112.stdout:client24973 2024-04-25T16:38:25.060 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a7905982-0320-11ef-bc93-c7b262605968.client24973/status') 2024-04-25T16:38:25.095 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:24 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:24.833+0000 7fb5c6bb8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T16:38:25.095 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:25.057+0000 7fb5c6bb8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:38:25.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T16:38:25.117 INFO:teuthology.orchestra.run.smithi112.stdout:instance: client.24973 (3)172.21.15.112:0/1041156082 2024-04-25T16:38:25.117 INFO:teuthology.orchestra.run.smithi112.stdout:blocklisted: false 2024-04-25T16:38:25.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records in 2024-04-25T16:38:25.117 INFO:teuthology.orchestra.run.smithi112.stderr:0+1 records out 2024-04-25T16:38:25.118 INFO:teuthology.orchestra.run.smithi112.stderr:72 bytes copied, 2.3213e-05 s, 3.1 MB/s 2024-04-25T16:38:25.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T16:38:25.125 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T16:38:25.134 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T16:38:25.134 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:38:25.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T16:38:25.199 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:24.820+0000 7fb862a83200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:38:25.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:24.780+0000 7f5e37d15200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 -- 172.21.15.112:0/119791484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 msgr2=0x7fbef0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 --2- 172.21.15.112:0/119791484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0104070 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009900 tx=0x7fbed802a410 comp rx=0 tx=0).stop 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 -- 172.21.15.112:0/119791484 shutdown_connections 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 --2- 172.21.15.112:0/119791484 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef0108960 0x7fbef010ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 --2- 172.21.15.112:0/119791484 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0104070 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 --2- 172.21.15.112:0/119791484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbef0102a20 0x7fbef0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 -- 172.21.15.112:0/119791484 >> 172.21.15.112:0/119791484 conn(0x7fbef00fdc10 msgr2=0x7fbef01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 -- 172.21.15.112:0/119791484 shutdown_connections 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.282+0000 7fbef5451640 1 -- 172.21.15.112:0/119791484 wait complete. 2024-04-25T16:38:25.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 Processor -- start 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 -- start start 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef0102a20 0x7fbef0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbef0108960 0x7fbef01a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010e550 con 0x7fbef0103c60 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010e3d0 con 0x7fbef0108960 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbef5451640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbef010e6d0 con 0x7fbef0102a20 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40226/0 (socket says 172.21.15.112:40226) 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbee7fff640 1 -- 172.21.15.112:0/757697794 learned_addr learned my addr 172.21.15.112:0/757697794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbeef7fe640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbef0108960 0x7fbef01a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbeeeffd640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef0102a20 0x7fbef0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbee7fff640 1 -- 172.21.15.112:0/757697794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef0102a20 msgr2=0x7fbef0199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.283+0000 7fbee7fff640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef0102a20 0x7fbef0199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbee7fff640 1 -- 172.21.15.112:0/757697794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbef0108960 msgr2=0x7fbef01a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbee7fff640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbef0108960 0x7fbef01a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbee7fff640 1 -- 172.21.15.112:0/757697794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef01a0c40 con 0x7fbef0103c60 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbee7fff640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0199640 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006010 tx=0x7fbed8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed803f030 con 0x7fbef0103c60 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed8003aa0 con 0x7fbef0103c60 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef01a0ed0 con 0x7fbef0103c60 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8035070 con 0x7fbef0103c60 2024-04-25T16:38:25.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.284+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01a12e0 con 0x7fbef0103c60 2024-04-25T16:38:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.285+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbef0107770 con 0x7fbef0103c60 2024-04-25T16:38:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.287+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 19) ==== 147538+0+0 (secure 0 0 0) 0x7fbed8042030 con 0x7fbef0103c60 2024-04-25T16:38:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.287+0000 7fbeecff9640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 0x7fbedc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.287+0000 7fbeeeffd640 1 -- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 msgr2=0x7fbedc0b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.287+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7fbed80ff890 con 0x7fbef0103c60 2024-04-25T16:38:25.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.287+0000 7fbeeeffd640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 0x7fbedc0b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:38:25.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.291+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbed80c7de0 con 0x7fbef0103c60 2024-04-25T16:38:25.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.488+0000 7fbeeeffd640 1 -- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 msgr2=0x7fbedc0b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:25.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.488+0000 7fbeeeffd640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 0x7fbedc0b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:38:25.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.514+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fbef002db50 con 0x7fbef0103c60 2024-04-25T16:38:25.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.566+0000 7fbeecff9640 1 -- 172.21.15.112:0/757697794 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7fbed80cc0e0 con 0x7fbef0103c60 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 msgr2=0x7fbedc0b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 0x7fbedc0b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 msgr2=0x7fbef0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0199640 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006010 tx=0x7fbed8031a50 comp rx=0 tx=0).stop 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 shutdown_connections 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbef0108960 0x7fbef01a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbef0103c60 0x7fbef0199640 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7fbedc0afb30 0x7fbedc0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.569+0000 7fbef5451640 1 --2- 172.21.15.112:0/757697794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef0102a20 0x7fbef0199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.570+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 >> 172.21.15.112:0/757697794 conn(0x7fbef00fdc10 msgr2=0x7fbef0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:25.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.570+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 shutdown_connections 2024-04-25T16:38:25.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.570+0000 7fbef5451640 1 -- 172.21.15.112:0/757697794 wait complete. 2024-04-25T16:38:25.569 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:25.305+0000 7f5e37d15200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:38:25.569 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:25.399+0000 7f5e37d15200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:38:25.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T16:38:25.691 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:25 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/757697794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T16:38:25.691 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:25.349+0000 7fb862a83200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:38:25.691 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:25.443+0000 7fb862a83200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 -- 172.21.15.112:0/2404805340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 msgr2=0x7f29a410abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 --2- 172.21.15.112:0/2404805340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a410abf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f299c009ca0 tx=0x7f299c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 -- 172.21.15.112:0/2404805340 shutdown_connections 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 --2- 172.21.15.112:0/2404805340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a41039b0 0x7f29a4103e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 --2- 172.21.15.112:0/2404805340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a410abf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 --2- 172.21.15.112:0/2404805340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29a4102fd0 0x7f29a41033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.696+0000 7f29abbb7640 1 -- 172.21.15.112:0/2404805340 >> 172.21.15.112:0/2404805340 conn(0x7f29a40fd390 msgr2=0x7f29a40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:25.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 -- 172.21.15.112:0/2404805340 shutdown_connections 2024-04-25T16:38:25.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 -- 172.21.15.112:0/2404805340 wait complete. 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 Processor -- start 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 -- start start 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a4102fd0 0x7f29a41022d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29a41039b0 0x7f29a419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a41a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29a410e550 con 0x7f29a41044b0 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f29a410e3d0 con 0x7f29a41039b0 2024-04-25T16:38:25.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.697+0000 7f29abbb7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f29a410e6d0 con 0x7f29a4102fd0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29aa154640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a41a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a4102fd0 0x7f29a41022d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29aa154640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a41a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47292/0 (socket says 172.21.15.112:47292) 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29aa154640 1 -- 172.21.15.112:0/886817533 learned_addr learned my addr 172.21.15.112:0/886817533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29a41039b0 0x7f29a419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29a41039b0 msgr2=0x7f29a419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29a41039b0 0x7f29a419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 msgr2=0x7f29a41a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a41a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 -- 172.21.15.112:0/886817533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a41a1ce0 con 0x7f29a4102fd0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29aa154640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29a41044b0 0x7f29a41a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29a9953640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a4102fd0 0x7f29a41022d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f29a406a2f0 tx=0x7f29a000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a000cd00 con 0x7f29a4102fd0 2024-04-25T16:38:25.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a41a1fd0 con 0x7f29a4102fd0 2024-04-25T16:38:25.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29a00108e0 con 0x7f29a4102fd0 2024-04-25T16:38:25.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00187c0 con 0x7f29a4102fd0 2024-04-25T16:38:25.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.698+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a2490 con 0x7f29a4102fd0 2024-04-25T16:38:25.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.699+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a4109980 con 0x7f29a4102fd0 2024-04-25T16:38:25.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.701+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 19) ==== 147538+0+0 (secure 0 0 0) 0x7f29a001f020 con 0x7f29a4102fd0 2024-04-25T16:38:25.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.701+0000 7f299affd640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 0x7f29900b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:25.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.701+0000 7f29a9152640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 msgr2=0x7f29900b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:25.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.701+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8298+0+0 (secure 0 0 0) 0x7f29a00df370 con 0x7f29a4102fd0 2024-04-25T16:38:25.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.701+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 0x7f29900b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T16:38:25.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.705+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29a00a7f10 con 0x7f29a4102fd0 2024-04-25T16:38:25.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:25 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/757697794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T16:38:25.845 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:25.617+0000 7fb5c6bb8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T16:38:25.845 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:25.720+0000 7fb5c6bb8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T16:38:25.876 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/757697794' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T16:38:25.876 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:25.647+0000 7f5e37d15200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:38:25.876 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:25.782+0000 7f5e37d15200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:38:25.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.902+0000 7f29a9152640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 msgr2=0x7f29900b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:25.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.902+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 0x7f29900b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T16:38:25.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:25.917+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 --> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f29a4063140 con 0x7f29900afb30 2024-04-25T16:38:25.949 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:25.692+0000 7fb862a83200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:38:25.949 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:25.830+0000 7fb862a83200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:38:25.949 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:25.926+0000 7fb862a83200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:38:26.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:25.878+0000 7f5e37d15200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:38:26.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:25.994+0000 7f5e37d15200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:26.303+0000 7f29a9152640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 msgr2=0x7f29900b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:26.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:26.303+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 0x7f29900b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T16:38:26.338 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:25.979+0000 7fb5c6bb8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T16:38:26.338 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:26 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:26.117+0000 7fb5c6bb8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T16:38:26.338 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:26 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:26.216+0000 7fb5c6bb8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T16:38:26.338 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:26 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:26.339+0000 7fb5c6bb8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:38:26.353 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:26.046+0000 7fb862a83200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T16:38:26.353 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:26.355+0000 7fb862a83200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:38:26.570 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:26.297+0000 7f5e37d15200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:38:26.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:26 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/757697794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T16:38:26.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:26 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:38:26.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:26 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/757697794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T16:38:26.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:26 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:38:26.902 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:26 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/757697794' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T16:38:26.940 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:26 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:38:27.095 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:26 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:26.647+0000 7fb5c6bb8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T16:38:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:27.104+0000 7f29a9152640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 msgr2=0x7f29900b2040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.112:6800/3947590362 2024-04-25T16:38:27.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:27.104+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 0x7f29900b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T16:38:27.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:26.903+0000 7f5e37d15200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:38:27.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y[34094]: 2024-04-25T16:38:27.125+0000 7f5e37d15200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:38:27.449 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:26.962+0000 7fb862a83200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:38:27.449 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z[34460]: 2024-04-25T16:38:27.185+0000 7fb862a83200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:38:27.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:27.513+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 20) ==== 147205+0+0 (secure 0 0 0) 0x7f29a0010a90 con 0x7f29a4102fd0 2024-04-25T16:38:27.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:27.514+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 msgr2=0x7f29900b2040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T16:38:27.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:27.514+0000 7f299affd640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/3947590362,v1:172.21.15.112:6801/3947590362] conn(0x7f29900afb30 0x7f29900b2040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:27.572 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.271+0000 7fb5c6bb8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T16:38:27.573 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.499+0000 7fb5c6bb8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T16:38:27.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Standby manager daemon y restarted 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Standby manager daemon y started 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Standby manager daemon z restarted 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Standby manager daemon z started 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:38:27.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Active manager daemon x restarted 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Activating manager daemon x 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: mgrmap e20: x(active, starting, since 0.0142217s), standbys: z, y 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:38:27.847 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb525ae9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb525ae9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb525ae9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb525ae9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb525ae9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb527aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb527aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb527aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.848 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb527aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:27.750+0000 7fb527aed700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:38:27.849 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T16:38:27.850 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:27 smithi112 ceph-mon[29122]: Manager daemon x is now available 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Standby manager daemon y restarted 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Standby manager daemon y started 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Standby manager daemon z restarted 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Standby manager daemon z started 2024-04-25T16:38:27.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Active manager daemon x restarted 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Activating manager daemon x 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: mgrmap e20: x(active, starting, since 0.0142217s), standbys: z, y 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T16:38:27.951 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:38:27.952 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T16:38:27.953 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:27 smithi133 ceph-mon[32691]: Manager daemon x is now available 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Standby manager daemon y restarted 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Standby manager daemon y started 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.116:0/1807512917' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:38:28.009 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Standby manager daemon z restarted 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Standby manager daemon z started 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.? 172.21.15.133:0/1933659076' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Active manager daemon x restarted 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Activating manager daemon x 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: osdmap e80: 12 total, 12 up, 12 in 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: mgrmap e20: x(active, starting, since 0.0142217s), standbys: z, y 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T16:38:28.010 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T16:38:28.011 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T16:38:28.012 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T16:38:28.013 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:27 smithi116 ceph-mon[32712]: Manager daemon x is now available 2024-04-25T16:38:28.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.521+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mon.1 v2:172.21.15.133:3300/0 8 ==== mgrmap(e 21) ==== 147332+0+0 (secure 0 0 0) 0x7f29a00a3aa0 con 0x7f29a4102fd0 2024-04-25T16:38:28.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.522+0000 7f299affd640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f29900735f0 0x7f2990073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:28.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.522+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f29900413a0 con 0x7f29900735f0 2024-04-25T16:38:28.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.524+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f29900735f0 0x7f2990073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:28.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.526+0000 7f29a9152640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f29900735f0 0x7f2990073a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f29940095f0 tx=0x7f2994005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:28.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.542+0000 7f299affd640 1 -- 172.21.15.112:0/886817533 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f29900413a0 con 0x7f29900735f0 2024-04-25T16:38:28.541 INFO:teuthology.orchestra.run.smithi112.stdout:{} 2024-04-25T16:38:28.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.544+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f29900735f0 msgr2=0x7f2990073a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.544+0000 7f29abbb7640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f29900735f0 0x7f2990073a00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f29940095f0 tx=0x7f2994005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:28.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.544+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a4102fd0 msgr2=0x7f29a41022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.544+0000 7f29abbb7640 1 --2- 172.21.15.112:0/886817533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f29a4102fd0 0x7f29a41022d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f29a406a2f0 tx=0x7f29a000ee50 comp rx=0 tx=0).stop 2024-04-25T16:38:28.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.544+0000 7f29aa154640 1 -- 172.21.15.112:0/886817533 reap_dead start 2024-04-25T16:38:28.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.545+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 shutdown_connections 2024-04-25T16:38:28.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.545+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 >> 172.21.15.112:0/886817533 conn(0x7f29a40fd390 msgr2=0x7f29a4106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:28.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.545+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 shutdown_connections 2024-04-25T16:38:28.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.545+0000 7f29abbb7640 1 -- 172.21.15.112:0/886817533 wait complete. 2024-04-25T16:38:28.554 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T16:38:28.567 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T16:38:28.584 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T16:38:28.584 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:38:28.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 -- 172.21.15.112:0/384080548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4103c50 msgr2=0x7efef4104060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 --2- 172.21.15.112:0/384080548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4103c50 0x7efef4104060 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efee4009900 tx=0x7efee402a410 comp rx=0 tx=0).stop 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 -- 172.21.15.112:0/384080548 shutdown_connections 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 --2- 172.21.15.112:0/384080548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef4108950 0x7efef410ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 --2- 172.21.15.112:0/384080548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4103c50 0x7efef4104060 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 --2- 172.21.15.112:0/384080548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef4102a10 0x7efef4103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.697 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 -- 172.21.15.112:0/384080548 >> 172.21.15.112:0/384080548 conn(0x7efef40fdc00 msgr2=0x7efef41000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 -- 172.21.15.112:0/384080548 shutdown_connections 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.696+0000 7efef9e20640 1 -- 172.21.15.112:0/384080548 wait complete. 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 Processor -- start 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 -- start start 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 0x7efef41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef4103c50 0x7efef41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef4108950 0x7efef41a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efef410e540 con 0x7efef4102a10 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efef410e3c0 con 0x7efef4108950 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef9e20640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efef410e6c0 con 0x7efef4103c50 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 0x7efef41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 0x7efef41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47444/0 (socket says 172.21.15.112:47444) 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef37fe640 1 -- 172.21.15.112:0/3574125119 learned_addr learned my addr 172.21.15.112:0/3574125119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:28.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.697+0000 7efef2ffd640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef4103c50 0x7efef41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef3fff640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef4108950 0x7efef41a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef37fe640 1 -- 172.21.15.112:0/3574125119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef4103c50 msgr2=0x7efef41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef37fe640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef4103c50 0x7efef41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef37fe640 1 -- 172.21.15.112:0/3574125119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef4108950 msgr2=0x7efef41a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef37fe640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef4108950 0x7efef41a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef37fe640 1 -- 172.21.15.112:0/3574125119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef41a0c70 con 0x7efef4102a10 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef37fe640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 0x7efef41991a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efee800e9f0 tx=0x7efee800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee800ccc0 con 0x7efef4102a10 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee80035f0 con 0x7efef4102a10 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef41a0f60 con 0x7efef4102a10 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee800ce60 con 0x7efef4102a10 2024-04-25T16:38:28.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.698+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41a13c0 con 0x7efef4102a10 2024-04-25T16:38:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.699+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef4050740 con 0x7efef4102a10 2024-04-25T16:38:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.701+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 147332+0+0 (secure 0 0 0) 0x7efee8003aa0 con 0x7efef4102a10 2024-04-25T16:38:28.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.701+0000 7efef0ff9640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efedc0af8e0 0x7efedc0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.701+0000 7efef2ffd640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efedc0af8e0 0x7efedc0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.701+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7efee80dedc0 con 0x7efef4102a10 2024-04-25T16:38:28.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.702+0000 7efef2ffd640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efedc0af8e0 0x7efedc0b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efee4009230 tx=0x7efee4031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:28.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.705+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efee80a7b50 con 0x7efef4102a10 2024-04-25T16:38:28.841 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:28 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:38:28.841 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:28 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T16:38:28.841 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:28 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T16:38:28.841 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:28 smithi133 ceph-mon[32691]: [25/Apr/2024:16:38:28] ENGINE Bus STARTING 2024-04-25T16:38:28.841 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:28 smithi133 ceph-mon[32691]: [25/Apr/2024:16:38:28] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T16:38:28.841 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:28 smithi133 ceph-mon[32691]: mgrmap e21: x(active, since 1.02401s), standbys: z, y 2024-04-25T16:38:28.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:38:28.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T16:38:28.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T16:38:28.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-mon[29122]: [25/Apr/2024:16:38:28] ENGINE Bus STARTING 2024-04-25T16:38:28.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-mon[29122]: [25/Apr/2024:16:38:28] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T16:38:28.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-mon[29122]: mgrmap e21: x(active, since 1.02401s), standbys: z, y 2024-04-25T16:38:28.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:28.531+0000 7fb522ae3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:28.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:28.531+0000 7fb522ae3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:28.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:28.531+0000 7fb522ae3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:28.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:28.531+0000 7fb522ae3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:28.846 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:38:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:38:28.531+0000 7fb522ae3700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:38:28.888 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:28 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:38:28.888 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:28 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T16:38:28.888 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:28 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T16:38:28.888 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:28 smithi116 ceph-mon[32712]: [25/Apr/2024:16:38:28] ENGINE Bus STARTING 2024-04-25T16:38:28.888 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:28 smithi116 ceph-mon[32712]: [25/Apr/2024:16:38:28] ENGINE Serving on https://172.21.15.112:7150 2024-04-25T16:38:28.888 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:28 smithi116 ceph-mon[32712]: mgrmap e21: x(active, since 1.02401s), standbys: z, y 2024-04-25T16:38:28.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.893+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7efef41077b0 con 0x7efef4102a10 2024-04-25T16:38:28.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.897+0000 7efef0ff9640 1 -- 172.21.15.112:0/3574125119 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v26)=0 v26) ==== 129+0+0 (secure 0 0 0) 0x7efef41077b0 con 0x7efef4102a10 2024-04-25T16:38:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efedc0af8e0 msgr2=0x7efedc0b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efedc0af8e0 0x7efedc0b1df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efee4009230 tx=0x7efee4031400 comp rx=0 tx=0).stop 2024-04-25T16:38:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 msgr2=0x7efef41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 0x7efef41991a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efee800e9f0 tx=0x7efee800eeb0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 shutdown_connections 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efedc0af8e0 0x7efedc0b1df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efef4103c50 0x7efef41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef4108950 0x7efef41a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 --2- 172.21.15.112:0/3574125119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efef4102a10 0x7efef41991a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 >> 172.21.15.112:0/3574125119 conn(0x7efef40fdc00 msgr2=0x7efef4104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 shutdown_connections 2024-04-25T16:38:28.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:28.900+0000 7efef9e20640 1 -- 172.21.15.112:0/3574125119 wait complete. 2024-04-25T16:38:28.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 -- 172.21.15.112:0/823569661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181022a0 msgr2=0x7fd91810bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 --2- 172.21.15.112:0/823569661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181022a0 0x7fd91810bd00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ca0 tx=0x7fd91002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 -- 172.21.15.112:0/823569661 shutdown_connections 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 --2- 172.21.15.112:0/823569661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9181017a0 0x7fd918101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 --2- 172.21.15.112:0/823569661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181022a0 0x7fd91810bd00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 --2- 172.21.15.112:0/823569661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9181050f0 0x7fd918105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 -- 172.21.15.112:0/823569661 >> 172.21.15.112:0/823569661 conn(0x7fd9180fd3c0 msgr2=0x7fd9180ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 -- 172.21.15.112:0/823569661 shutdown_connections 2024-04-25T16:38:29.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.023+0000 7fd91c805640 1 -- 172.21.15.112:0/823569661 wait complete. 2024-04-25T16:38:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 Processor -- start 2024-04-25T16:38:29.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 -- start start 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 0x7fd91819a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9181022a0 0x7fd91819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9181050f0 0x7fd9181a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810f660 con 0x7fd9181017a0 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810f4e0 con 0x7fd9181022a0 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91c805640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd91810f7e0 con 0x7fd9181050f0 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 0x7fd91819a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 0x7fd91819a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47462/0 (socket says 172.21.15.112:47462) 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.024+0000 7fd91659c640 1 -- 172.21.15.112:0/3208138860 learned_addr learned my addr 172.21.15.112:0/3208138860 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd916d9d640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9181050f0 0x7fd9181a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd90ffff640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9181022a0 0x7fd91819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91659c640 1 -- 172.21.15.112:0/3208138860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9181050f0 msgr2=0x7fd9181a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91659c640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9181050f0 0x7fd9181a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91659c640 1 -- 172.21.15.112:0/3208138860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9181022a0 msgr2=0x7fd91819a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91659c640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9181022a0 0x7fd91819a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91659c640 1 -- 172.21.15.112:0/3208138860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9181a1cd0 con 0x7fd9181017a0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91659c640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 0x7fd91819a1e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd908005fd0 tx=0x7fd90800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908017070 con 0x7fd9181017a0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9181a1fc0 con 0x7fd9181017a0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd908009c30 con 0x7fd9181017a0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908003d10 con 0x7fd9181017a0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.025+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a2420 con 0x7fd9181017a0 2024-04-25T16:38:29.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.026+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd918050740 con 0x7fd9181017a0 2024-04-25T16:38:29.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.028+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 147332+0+0 (secure 0 0 0) 0x7fd908003eb0 con 0x7fd9181017a0 2024-04-25T16:38:29.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.028+0000 7fd90f7fe640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd9040af8e0 0x7fd9040b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.028+0000 7fd90ffff640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd9040af8e0 0x7fd9040b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.028+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fd9080ded30 con 0x7fd9181017a0 2024-04-25T16:38:29.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.029+0000 7fd90ffff640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd9040af8e0 0x7fd9040b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd900005d00 tx=0x7fd900005c70 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:29.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.032+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd908020030 con 0x7fd9181017a0 2024-04-25T16:38:29.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.212+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fd91810aae0 con 0x7fd9181017a0 2024-04-25T16:38:29.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.217+0000 7fd90f7fe640 1 -- 172.21.15.112:0/3208138860 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) ==== 138+0+0 (secure 0 0 0) 0x7fd91810aae0 con 0x7fd9181017a0 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd9040af8e0 msgr2=0x7fd9040b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd9040af8e0 0x7fd9040b1df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd900005d00 tx=0x7fd900005c70 comp rx=0 tx=0).stop 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 msgr2=0x7fd91819a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 0x7fd91819a1e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd908005fd0 tx=0x7fd90800dde0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 shutdown_connections 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd9040af8e0 0x7fd9040b1df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9181050f0 0x7fd9181a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9181022a0 0x7fd91819a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 --2- 172.21.15.112:0/3208138860 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9181017a0 0x7fd91819a1e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 >> 172.21.15.112:0/3208138860 conn(0x7fd9180fd3c0 msgr2=0x7fd918107c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 shutdown_connections 2024-04-25T16:38:29.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.219+0000 7fd91c805640 1 -- 172.21.15.112:0/3208138860 wait complete. 2024-04-25T16:38:29.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 -- 172.21.15.112:0/278036022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 msgr2=0x7f2aa810ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 --2- 172.21.15.112:0/278036022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa810ad80 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009ca0 tx=0x7f2aa002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 -- 172.21.15.112:0/278036022 shutdown_connections 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 --2- 172.21.15.112:0/278036022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 0x7f2aa8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 --2- 172.21.15.112:0/278036022 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa810ad80 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 --2- 172.21.15.112:0/278036022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 0x7f2aa8103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 -- 172.21.15.112:0/278036022 >> 172.21.15.112:0/278036022 conn(0x7f2aa80fdc10 msgr2=0x7f2aa81000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 -- 172.21.15.112:0/278036022 shutdown_connections 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.353+0000 7f2aaf425640 1 -- 172.21.15.112:0/278036022 wait complete. 2024-04-25T16:38:29.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 Processor -- start 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 -- start start 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 0x7f2aa819a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 0x7f2aa819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa81a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa810e550 con 0x7f2aa8108960 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa810e3d0 con 0x7f2aa8102a20 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aaf425640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa810e6d0 con 0x7f2aa8103c60 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad9c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa81a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad1c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 0x7f2aa819a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad9c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa81a14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47488/0 (socket says 172.21.15.112:47488) 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad9c2640 1 -- 172.21.15.112:0/3020781158 learned_addr learned my addr 172.21.15.112:0/3020781158 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aac9c0640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 0x7f2aa819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad1c1640 1 -- 172.21.15.112:0/3020781158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 msgr2=0x7f2aa819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad1c1640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 0x7f2aa819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad1c1640 1 -- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 msgr2=0x7f2aa81a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad1c1640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa81a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aad1c1640 1 -- 172.21.15.112:0/3020781158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa81a1b40 con 0x7f2aa8102a20 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.354+0000 7f2aac9c0640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 0x7f2aa819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.355+0000 7f2aad1c1640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 0x7f2aa819a000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2aa81022f0 tx=0x7f2aa400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.355+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400cd00 con 0x7f2aa8102a20 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.355+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa81a1e30 con 0x7f2aa8102a20 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.355+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa40108e0 con 0x7f2aa8102a20 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.355+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40187c0 con 0x7f2aa8102a20 2024-04-25T16:38:29.356 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.355+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa81a22a0 con 0x7f2aa8102a20 2024-04-25T16:38:29.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.356+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa8108060 con 0x7f2aa8102a20 2024-04-25T16:38:29.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.357+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 21) ==== 147332+0+0 (secure 0 0 0) 0x7f2aa401f020 con 0x7f2aa8102a20 2024-04-25T16:38:29.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.358+0000 7f2a9e7fc640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f2a940af8e0 0x7f2a940b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.358+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f2aa40de880 con 0x7f2aa8102a20 2024-04-25T16:38:29.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.358+0000 7f2aac9c0640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f2a940af8e0 0x7f2a940b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.359+0000 7f2aac9c0640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f2a940af8e0 0x7f2a940b1df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2aa819b0a0 tx=0x7f2a98005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:29.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.363+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2aa40a7610 con 0x7f2aa8102a20 2024-04-25T16:38:29.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.589+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f2aa8062fe0 con 0x7f2a940af8e0 2024-04-25T16:38:29.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.628+0000 7f2a9e7fc640 1 -- 172.21.15.112:0/3020781158 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2aa8062fe0 con 0x7f2a940af8e0 2024-04-25T16:38:29.627 INFO:teuthology.orchestra.run.smithi112.stdout:Schedule set for path / 2024-04-25T16:38:29.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f2a940af8e0 msgr2=0x7f2a940b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f2a940af8e0 0x7f2a940b1df0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2aa819b0a0 tx=0x7f2a98005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 msgr2=0x7f2aa819a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 0x7f2aa819a000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2aa81022f0 tx=0x7f2aa400ee50 comp rx=0 tx=0).stop 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 shutdown_connections 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f2a940af8e0 0x7f2a940b1df0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2aa8103c60 0x7f2aa819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2aa8108960 0x7f2aa81a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 --2- 172.21.15.112:0/3020781158 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa8102a20 0x7f2aa819a000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 >> 172.21.15.112:0/3020781158 conn(0x7f2aa80fdc10 msgr2=0x7f2aa806b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 shutdown_connections 2024-04-25T16:38:29.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.632+0000 7f2aaf425640 1 -- 172.21.15.112:0/3020781158 wait complete. 2024-04-25T16:38:29.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 -- 172.21.15.112:0/1759158270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83901097d0 msgr2=0x7f839010bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 --2- 172.21.15.112:0/1759158270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83901097d0 0x7f839010bce0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8388009ca0 tx=0x7f838802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 -- 172.21.15.112:0/1759158270 shutdown_connections 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 --2- 172.21.15.112:0/1759158270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8390106c60 0x7f8390109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 --2- 172.21.15.112:0/1759158270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83901097d0 0x7f839010bce0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 --2- 172.21.15.112:0/1759158270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390101e80 0x7f8390102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 -- 172.21.15.112:0/1759158270 >> 172.21.15.112:0/1759158270 conn(0x7f83900fd390 msgr2=0x7f83900ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 -- 172.21.15.112:0/1759158270 shutdown_connections 2024-04-25T16:38:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.772+0000 7f83987c7640 1 -- 172.21.15.112:0/1759158270 wait complete. 2024-04-25T16:38:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 Processor -- start 2024-04-25T16:38:29.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 -- start start 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8390101e80 0x7f8390104350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 0x7f83901029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83901097d0 0x7f8390102f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f8395d62640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 0x7f83901029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f8395d62640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 0x7f83901029f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:51784/0 (socket says 172.21.15.112:51784) 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f839010f6b0 con 0x7f8390101e80 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f8396563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8390101e80 0x7f8390104350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f839010f530 con 0x7f8390106c60 2024-04-25T16:38:29.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f8396563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8390101e80 0x7f8390104350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47524/0 (socket says 172.21.15.112:47524) 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f83987c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f839010f830 con 0x7f83901097d0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.773+0000 7f8396563640 1 -- 172.21.15.112:0/276147697 learned_addr learned my addr 172.21.15.112:0/276147697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83901097d0 0x7f8390102f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 -- 172.21.15.112:0/276147697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 msgr2=0x7f83901029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 0x7f83901029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 -- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8390101e80 msgr2=0x7f8390104350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8390101e80 0x7f8390104350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 -- 172.21.15.112:0/276147697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8390103660 con 0x7f83901097d0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8395d62640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 0x7f83901029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f8396d64640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83901097d0 0x7f8390102f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f838802aeb0 tx=0x7f83880316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388003b70 con 0x7f83901097d0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83901a4280 con 0x7f83901097d0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8388003d10 con 0x7f83901097d0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.774+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838802fae0 con 0x7f83901097d0 2024-04-25T16:38:29.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.775+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83901a4740 con 0x7f83901097d0 2024-04-25T16:38:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.775+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83900feeb0 con 0x7f83901097d0 2024-04-25T16:38:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.777+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 21) ==== 147332+0+0 (secure 0 0 0) 0x7f8388004610 con 0x7f83901097d0 2024-04-25T16:38:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.778+0000 7f83877fe640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f837c0af8e0 0x7f837c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:29.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.778+0000 7f8396563640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f837c0af8e0 0x7f837c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:29.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.778+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f83880fffc0 con 0x7f83901097d0 2024-04-25T16:38:29.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.778+0000 7f8396563640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f837c0af8e0 0x7f837c0b1df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f838c0058c0 tx=0x7f838c005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:29.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.782+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83880c8d10 con 0x7f83901097d0 2024-04-25T16:38:29.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:29.995+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8390062fe0 con 0x7f837c0af8e0 2024-04-25T16:38:30.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.023+0000 7f83877fe640 1 -- 172.21.15.112:0/276147697 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8390062fe0 con 0x7f837c0af8e0 2024-04-25T16:38:30.022 INFO:teuthology.orchestra.run.smithi112.stdout:Retention added to path / 2024-04-25T16:38:30.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f837c0af8e0 msgr2=0x7f837c0b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.024 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f837c0af8e0 0x7f837c0b1df0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f838c0058c0 tx=0x7f838c005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83901097d0 msgr2=0x7f8390102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83901097d0 0x7f8390102f30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f838802aeb0 tx=0x7f83880316f0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 shutdown_connections 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f837c0af8e0 0x7f837c0b1df0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83901097d0 0x7f8390102f30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8390106c60 0x7f83901029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 --2- 172.21.15.112:0/276147697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8390101e80 0x7f8390104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 >> 172.21.15.112:0/276147697 conn(0x7f83900fd390 msgr2=0x7f83901077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 shutdown_connections 2024-04-25T16:38:30.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.025+0000 7f83987c7640 1 -- 172.21.15.112:0/276147697 wait complete. 2024-04-25T16:38:30.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T16:38:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 -- 172.21.15.112:0/708491010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e010bb30 msgr2=0x7f40e010df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 --2- 172.21.15.112:0/708491010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e010bb30 0x7f40e010df50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009ca0 tx=0x7f40d802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 -- 172.21.15.112:0/708491010 shutdown_connections 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 --2- 172.21.15.112:0/708491010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0075860 0x7f40e010b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 --2- 172.21.15.112:0/708491010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e010bb30 0x7f40e010df50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 --2- 172.21.15.112:0/708491010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40e0074f10 0x7f40e0075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 -- 172.21.15.112:0/708491010 >> 172.21.15.112:0/708491010 conn(0x7f40e00fed30 msgr2=0x7f40e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 -- 172.21.15.112:0/708491010 shutdown_connections 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.146+0000 7f40e5e53640 1 -- 172.21.15.112:0/708491010 wait complete. 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 Processor -- start 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 -- start start 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0074f10 0x7f40e019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 0x7f40e019c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40e010bb30 0x7f40e01a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40e0111860 con 0x7f40e0075860 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40e01116e0 con 0x7f40e010bb30 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.147+0000 7f40e5e53640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40e01119e0 con 0x7f40e0074f10 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 0x7f40e019c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 0x7f40e019c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47538/0 (socket says 172.21.15.112:47538) 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 -- 172.21.15.112:0/2373630648 learned_addr learned my addr 172.21.15.112:0/2373630648 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40df7fe640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0074f10 0x7f40e019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40dffff640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40e010bb30 0x7f40e01a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 -- 172.21.15.112:0/2373630648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0074f10 msgr2=0x7f40e019c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0074f10 0x7f40e019c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 -- 172.21.15.112:0/2373630648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40e010bb30 msgr2=0x7f40e01a3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40e010bb30 0x7f40e01a3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 -- 172.21.15.112:0/2373630648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e01a3ec0 con 0x7f40e0075860 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40deffd640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 0x7f40e019c8c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e9f0 tx=0x7f40d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d000cd80 con 0x7f40e0075860 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e01a41b0 con 0x7f40e0075860 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40d00054d0 con 0x7f40e0075860 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d0010690 con 0x7f40e0075860 2024-04-25T16:38:30.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.148+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e01a4700 con 0x7f40e0075860 2024-04-25T16:38:30.152 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.150+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40e010acf0 con 0x7f40e0075860 2024-04-25T16:38:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.151+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 21) ==== 147332+0+0 (secure 0 0 0) 0x7f40d0004e90 con 0x7f40e0075860 2024-04-25T16:38:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.151+0000 7f40dcff9640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40c80af8e0 0x7f40c80b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.151+0000 7f40df7fe640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40c80af8e0 0x7f40c80b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.151+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f40d00ded80 con 0x7f40e0075860 2024-04-25T16:38:30.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.152+0000 7f40df7fe640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40c80af8e0 0x7f40c80b1df0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40e0076740 tx=0x7f40d4005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:30.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.156+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40d00a7b10 con 0x7f40e0075860 2024-04-25T16:38:30.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: [25/Apr/2024:16:38:28] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T16:38:30.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: [25/Apr/2024:16:38:28] ENGINE Bus STARTED 2024-04-25T16:38:30.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3574125119' entity='client.admin' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3208138860' entity='client.admin' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.200 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:29 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.230+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mgrmap(e 22) ==== 147692+0+0 (secure 0 0 0) 0x7f40d00a36a0 con 0x7f40e0075860 2024-04-25T16:38:30.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: [25/Apr/2024:16:38:28] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: [25/Apr/2024:16:38:28] ENGINE Bus STARTED 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3574125119' entity='client.admin' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3208138860' entity='client.admin' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.259 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:29 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: [25/Apr/2024:16:38:28] ENGINE Serving on http://172.21.15.112:8765 2024-04-25T16:38:30.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: [25/Apr/2024:16:38:28] ENGINE Bus STARTED 2024-04-25T16:38:30.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:30.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3574125119' entity='client.admin' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3208138860' entity='client.admin' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.346 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:29 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:30.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.388+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f40e0062fe0 con 0x7f40c80af8e0 2024-04-25T16:38:30.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.395+0000 7f40dcff9640 1 -- 172.21.15.112:0/2373630648 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f40e0062fe0 con 0x7f40c80af8e0 2024-04-25T16:38:30.394 INFO:teuthology.orchestra.run.smithi112.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T16:38:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40c80af8e0 msgr2=0x7f40c80b1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40c80af8e0 0x7f40c80b1df0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f40e0076740 tx=0x7f40d4005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 msgr2=0x7f40e019c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 0x7f40e019c8c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f40d000e9f0 tx=0x7f40d000edf0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 shutdown_connections 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40c80af8e0 0x7f40c80b1df0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40e010bb30 0x7f40e01a3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40e0075860 0x7f40e019c8c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 --2- 172.21.15.112:0/2373630648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40e0074f10 0x7f40e019c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 >> 172.21.15.112:0/2373630648 conn(0x7f40e00fed30 msgr2=0x7f40e00ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:30.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 shutdown_connections 2024-04-25T16:38:30.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.398+0000 7f40e5e53640 1 -- 172.21.15.112:0/2373630648 wait complete. 2024-04-25T16:38:30.408 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T16:38:30.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 -- 172.21.15.112:0/4042918051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04107100 msgr2=0x7f3e04109520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 --2- 172.21.15.112:0/4042918051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04107100 0x7f3e04109520 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009900 tx=0x7f3df402a410 comp rx=0 tx=0).stop 2024-04-25T16:38:30.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 -- 172.21.15.112:0/4042918051 shutdown_connections 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 --2- 172.21.15.112:0/4042918051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e04109a60 0x7f3e0410be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 --2- 172.21.15.112:0/4042918051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04107100 0x7f3e04109520 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 --2- 172.21.15.112:0/4042918051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e04101f50 0x7f3e04106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 -- 172.21.15.112:0/4042918051 >> 172.21.15.112:0/4042918051 conn(0x7f3e040fd390 msgr2=0x7f3e040ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 -- 172.21.15.112:0/4042918051 shutdown_connections 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.522+0000 7f3e0a740640 1 -- 172.21.15.112:0/4042918051 wait complete. 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 Processor -- start 2024-04-25T16:38:30.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 -- start start 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 0x7f3e04104390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e04107100 0x7f3e04102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e04109a60 0x7f3e04102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0410f6e0 con 0x7f3e04101f50 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0410f560 con 0x7f3e04107100 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e0a740640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0410f860 con 0x7f3e04109a60 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 0x7f3e04104390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 0x7f3e04104390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47560/0 (socket says 172.21.15.112:47560) 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 -- 172.21.15.112:0/573710596 learned_addr learned my addr 172.21.15.112:0/573710596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e08cdd640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e04109a60 0x7f3e04102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e037fe640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e04107100 0x7f3e04102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 -- 172.21.15.112:0/573710596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e04109a60 msgr2=0x7f3e04102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e04109a60 0x7f3e04102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 -- 172.21.15.112:0/573710596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e04107100 msgr2=0x7f3e04102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e04107100 0x7f3e04102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.523+0000 7f3e03fff640 1 -- 172.21.15.112:0/573710596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041036a0 con 0x7f3e04101f50 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.524+0000 7f3e03fff640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 0x7f3e04104390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3df800e9a0 tx=0x7f3df800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.524+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df800cd80 con 0x7f3e04101f50 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.524+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df80035f0 con 0x7f3e04101f50 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.524+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e041a4370 con 0x7f3e04101f50 2024-04-25T16:38:30.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.524+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8010690 con 0x7f3e04101f50 2024-04-25T16:38:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.524+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a4830 con 0x7f3e04101f50 2024-04-25T16:38:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.525+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e040feeb0 con 0x7f3e04101f50 2024-04-25T16:38:30.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.528+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 22) ==== 147692+0+0 (secure 0 0 0) 0x7f3df8003aa0 con 0x7f3e04101f50 2024-04-25T16:38:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.528+0000 7f3e017fa640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3dec0afce0 0x7f3dec0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.528+0000 7f3e037fe640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3dec0afce0 0x7f3dec0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.528+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f3df80df610 con 0x7f3e04101f50 2024-04-25T16:38:30.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.529+0000 7f3e037fe640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3dec0afce0 0x7f3dec0b21f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009230 tx=0x7f3df4031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:30.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.532+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3df80a7d20 con 0x7f3e04101f50 2024-04-25T16:38:30.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.736+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f3e04062fe0 con 0x7f3dec0afce0 2024-04-25T16:38:30.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.744+0000 7f3e017fa640 1 -- 172.21.15.112:0/573710596 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f3e04062fe0 con 0x7f3dec0afce0 2024-04-25T16:38:30.743 INFO:teuthology.orchestra.run.smithi112.stdout:/ 1m 6m3h 2024-04-25T16:38:30.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3dec0afce0 msgr2=0x7f3dec0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3dec0afce0 0x7f3dec0b21f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009230 tx=0x7f3df4031400 comp rx=0 tx=0).stop 2024-04-25T16:38:30.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 msgr2=0x7f3e04104390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:30.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 0x7f3e04104390 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3df800e9a0 tx=0x7f3df800eda0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 shutdown_connections 2024-04-25T16:38:30.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3dec0afce0 0x7f3dec0b21f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e04109a60 0x7f3e04102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e04107100 0x7f3e04102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 --2- 172.21.15.112:0/573710596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e04101f50 0x7f3e04104390 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 >> 172.21.15.112:0/573710596 conn(0x7f3e040fd390 msgr2=0x7f3e041056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 shutdown_connections 2024-04-25T16:38:30.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:38:30.746+0000 7f3e0a740640 1 -- 172.21.15.112:0/573710596 wait complete. 2024-04-25T16:38:30.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T16:38:30.783 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T16:38:30.794 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T16:38:30.830 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:38:30.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T16:38:30.832 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T16:38:30.845 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:38:30.845 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T16:38:30.845 INFO:tasks.workunit:timeout=3h 2024-04-25T16:38:30.845 INFO:tasks.workunit:cleanup=True 2024-04-25T16:38:30.846 DEBUG:teuthology.orchestra.run.smithi112:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 16:36:32.667785776 +0000 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 16:36:32.667785776 +0000 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 16:38:24.796133746 +0000 2024-04-25T16:38:30.862 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2024-04-25 16:36:32.667785776 +0000 2024-04-25T16:38:30.863 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T16:38:30.863 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T16:38:30.931 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T16:38:30.986 INFO:tasks.workunit.client.0.smithi112.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T16:38:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:38:31.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:31 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:38:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T16:38:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:38:31.597 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:31 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:38:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='client.25091 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:38:31.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:31 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:38:32.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:32.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-25T16:38:32.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Unable to set osd_memory_target on smithi133 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T16:38:32.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:32.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi116 to 176.2M 2024-04-25T16:38:32.498 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Unable to set osd_memory_target on smithi116 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Unable to set osd_memory_target on smithi112 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.499 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:38:32.500 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:38:32.500 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:38:32.500 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-25T16:38:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Unable to set osd_memory_target on smithi133 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T16:38:32.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi116 to 176.2M 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Unable to set osd_memory_target on smithi116 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Unable to set osd_memory_target on smithi112 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:38:32.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Unable to set osd_memory_target on smithi133 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi116 to 176.2M 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Unable to set osd_memory_target on smithi116 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Unable to set osd_memory_target on smithi112 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Updating smithi112:/etc/ceph/ceph.conf 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Updating smithi116:/etc/ceph/ceph.conf 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Updating smithi133:/etc/ceph/ceph.conf 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:38:32.701 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:38:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:33 smithi116 ceph-mon[32712]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.510 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:33 smithi112 ceph-mon[29122]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T16:38:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: Updating smithi112:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: Updating smithi116:/etc/ceph/ceph.client.admin.keyring 2024-04-25T16:38:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T16:38:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: Updating smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: Updating smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: Updating smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T16:38:33.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:33 smithi133 ceph-mon[32691]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T16:38:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:35 smithi116 ceph-mon[32712]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T16:38:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:35 smithi112 ceph-mon[29122]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T16:38:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:35 smithi133 ceph-mon[32691]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T16:38:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:37 smithi116 ceph-mon[32712]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 133 KiB/s wr, 37 op/s 2024-04-25T16:38:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:37 smithi112 ceph-mon[29122]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 133 KiB/s wr, 37 op/s 2024-04-25T16:38:37.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:37 smithi133 ceph-mon[32691]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 133 KiB/s wr, 37 op/s 2024-04-25T16:38:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:39 smithi116 ceph-mon[32712]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 103 KiB/s wr, 28 op/s 2024-04-25T16:38:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:39 smithi112 ceph-mon[29122]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 103 KiB/s wr, 28 op/s 2024-04-25T16:38:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:39 smithi133 ceph-mon[32691]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 103 KiB/s wr, 28 op/s 2024-04-25T16:38:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:41 smithi116 ceph-mon[32712]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 85 KiB/s wr, 26 op/s 2024-04-25T16:38:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:41 smithi112 ceph-mon[29122]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 85 KiB/s wr, 26 op/s 2024-04-25T16:38:41.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:41 smithi133 ceph-mon[32691]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 85 KiB/s wr, 26 op/s 2024-04-25T16:38:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:43 smithi116 ceph-mon[32712]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 24 op/s 2024-04-25T16:38:43.552 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:43 smithi112 ceph-mon[29122]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 24 op/s 2024-04-25T16:38:43.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:43 smithi133 ceph-mon[32691]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 24 op/s 2024-04-25T16:38:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.983+0000 7f3a922f9640 1 -- 172.21.15.112:0/3767199374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 msgr2=0x7f3a8c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.983+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3767199374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c103c40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009900 tx=0x7f3a8002a410 comp rx=0 tx=0).stop 2024-04-25T16:38:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.983+0000 7f3a922f9640 1 -- 172.21.15.112:0/3767199374 shutdown_connections 2024-04-25T16:38:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.983+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3767199374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c1042b0 0x7f3a8c110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3767199374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c103c40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3767199374 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 0x7f3a8c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 -- 172.21.15.112:0/3767199374 >> 172.21.15.112:0/3767199374 conn(0x7f3a8c079170 msgr2=0x7f3a8c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 -- 172.21.15.112:0/3767199374 shutdown_connections 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 -- 172.21.15.112:0/3767199374 wait complete. 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 Processor -- start 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.984+0000 7f3a922f9640 1 -- start start 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a922f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c1042b0 0x7f3a8c19a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a922f9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 0x7f3a8c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a922f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a922f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c113da0 con 0x7f3a8c10a000 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a922f9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c113c20 con 0x7f3a8c109b60 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a922f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c113f20 con 0x7f3a8c1042b0 2024-04-25T16:38:43.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41526/0 (socket says 172.21.15.112:41526) 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 -- 172.21.15.112:0/3583144241 learned_addr learned my addr 172.21.15.112:0/3583144241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c1042b0 0x7f3a8c19a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 0x7f3a8c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 -- 172.21.15.112:0/3583144241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c1042b0 msgr2=0x7f3a8c19a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c1042b0 0x7f3a8c19a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 -- 172.21.15.112:0/3583144241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 msgr2=0x7f3a8c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 0x7f3a8c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 -- 172.21.15.112:0/3583144241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c19f5c0 con 0x7f3a8c10a000 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 0x7f3a8c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a90896640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c19ef80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400eaa0 tx=0x7f3a8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:43.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.985+0000 7f3a897fa640 1 -- 172.21.15.112:0/3583144241 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8400cdb0 con 0x7f3a8c10a000 2024-04-25T16:38:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.986+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c19f8b0 con 0x7f3a8c10a000 2024-04-25T16:38:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.986+0000 7f3a897fa640 1 -- 172.21.15.112:0/3583144241 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a84005020 con 0x7f3a8c10a000 2024-04-25T16:38:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.986+0000 7f3a897fa640 1 -- 172.21.15.112:0/3583144241 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840116d0 con 0x7f3a8c10a000 2024-04-25T16:38:43.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.986+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c19fd80 con 0x7f3a8c10a000 2024-04-25T16:38:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a897fa640 1 -- 172.21.15.112:0/3583144241 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 23) ==== 147748+0+0 (secure 0 0 0) 0x7f3a840051d0 con 0x7f3a8c10a000 2024-04-25T16:38:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 Processor -- start 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 -- start start 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853262980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 0x555853262ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853263400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5558531c9460 con 0x555852e9a2f0 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555853183590 con 0x555852ec31b0 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a991af740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558531c5420 con 0x555852ea24d0 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.988+0000 7f3a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 0x555853262ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 0x555853262ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41536/0 (socket says 172.21.15.112:41536) 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 -- 172.21.15.112:0/3493079419 learned_addr learned my addr 172.21.15.112:0/3493079419 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853263400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853262980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a897fa640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3a740afd70 0x7f3a740b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 -- 172.21.15.112:0/3493079419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 msgr2=0x555853263400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a90896640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3a740afd70 0x7f3a740b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853263400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 -- 172.21.15.112:0/3493079419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 msgr2=0x555853262980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853262980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 -- 172.21.15.112:0/3493079419 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558531900d0 con 0x555852e9a2f0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a897fa640 1 -- 172.21.15.112:0/3583144241 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f3a840df5c0 con 0x7f3a8c10a000 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 0x555853262ec0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009ba0 tx=0x7f3a7c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18d49d53fa2ad98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a727fc640 1 -- 172.21.15.112:0/3493079419 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c004020 con 0x555852e9a2f0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a727fc640 1 -- 172.21.15.112:0/3493079419 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c0041c0 con 0x555852e9a2f0 2024-04-25T16:38:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.989+0000 7f3a727fc640 1 -- 172.21.15.112:0/3493079419 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0126b0 con 0x555852e9a2f0 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 -- 172.21.15.112:0/3493079419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 msgr2=0x555853262ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a90896640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3a740afd70 0x7f3a740b2280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3a840dcbd0 tx=0x7f3a84074d50 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 0x555853262ec0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009ba0 tx=0x7f3a7c0117b0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 -- 172.21.15.112:0/3493079419 shutdown_connections 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853263400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852e9a2f0 0x555853262ec0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 --2- 172.21.15.112:0/3493079419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853262980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 -- 172.21.15.112:0/3493079419 >> 172.21.15.112:0/3493079419 conn(0x555852eb7300 msgr2=0x55585325f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 -- 172.21.15.112:0/3493079419 shutdown_connections 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.990+0000 7f3a991af740 1 -- 172.21.15.112:0/3493079419 wait complete. 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 -- start start 2024-04-25T16:38:43.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 Processor -- start 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x555853271540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853271a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853277510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555853274090 con 0x555852ea24d0 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555853150b60 con 0x555852ec31b0 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555853274210 con 0x555852e9a2f0 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a991af740 10 client.0 _ll_register_callbacks cb 0x5558531752b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853271a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853271a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41544/0 (socket says 172.21.15.112:41544) 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a8bfff640 1 -- 172.21.15.112:0/1769340093 learned_addr learned my addr 172.21.15.112:0/1769340093 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x555853271540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:43.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.991+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853277510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8bfff640 1 -- 172.21.15.112:0/1769340093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 msgr2=0x555853271540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x555853271540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8bfff640 1 -- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 msgr2=0x555853277510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853277510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8bfff640 1 -- 172.21.15.112:0/1769340093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555852edb210 con 0x555852ea24d0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853277510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853271a80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112f33cd1f273389 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/1769340093 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c01f040 con 0x555852ea24d0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a7c0041c0 con 0x555852ea24d0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/1769340093 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f3a7c0035f0 con 0x555852ea24d0 2024-04-25T16:38:44.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/1769340093 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c032050 con 0x555852ea24d0 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a991af740 10 client.15405 fetch_fsmap learned FSMap version 22 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555852fa7620 con 0x555852ea24d0 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/1769340093 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c0041c0 con 0x555852ea24d0 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/1769340093 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3a7c012bd0 con 0x555852ea24d0 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a991af740 10 client.15405 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.992+0000 7f3a991af740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a991af740 20 client.15405 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a991af740 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e81d80 0x5558532839b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a991af740 4 client.15405 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558531d7070 con 0x555852e81d80 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e81d80 0x5558532839b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a6b7fe640 10 client.15405 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.993+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e81d80 0x5558532839b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32853336c3b6aa05 server_cookie=9d671fa14501362f in_seq=0 out_seq=0 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/1769340093 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5558531d7070 con 0x555852e81d80 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a6b7fe640 10 client.15405 handle_command_reply: tid=0 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 client.15405 shutdown 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 msgr2=0x555853271a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853271a80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c005760 comp rx=0 tx=0).stop 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 shutdown_connections 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x555853277510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853271a80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e81d80 0x5558532839b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 --2- 172.21.15.112:0/1769340093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x555853271540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a6b7fe640 0 client.15405 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 >> 172.21.15.112:0/1769340093 conn(0x555852eb7300 msgr2=0x555853263330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 shutdown_connections 2024-04-25T16:38:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 1 -- 172.21.15.112:0/1769340093 wait complete. 2024-04-25T16:38:44.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:43.994+0000 7f3a991af740 20 client.15405 trim_cache size 0 max 16384 2024-04-25T16:38:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 Processor -- start 2024-04-25T16:38:44.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 -- start start 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852e9a2f0 0x555853277440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853270310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 0x55585326ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b0280 con 0x555852ec31b0 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1adb50 con 0x555852e9a2f0 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a991af740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1ab9a0 con 0x555852ea24d0 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 0x55585326ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 0x55585326ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41548/0 (socket says 172.21.15.112:41548) 2024-04-25T16:38:44.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 -- 172.21.15.112:0/3195030107 learned_addr learned my addr 172.21.15.112:0/3195030107 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 -- 172.21.15.112:0/3195030107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 msgr2=0x555853270310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853270310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 -- 172.21.15.112:0/3195030107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852e9a2f0 msgr2=0x555853277440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852e9a2f0 0x555853277440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.009+0000 7f3a8b7fe640 1 -- 172.21.15.112:0/3195030107 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585326f880 con 0x555852ec31b0 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.010+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 0x55585326ad30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a800096c0 tx=0x7f3a80031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40d8b861a534053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.010+0000 7f3a71ffb640 1 -- 172.21.15.112:0/3195030107 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80035040 con 0x555852ec31b0 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.010+0000 7f3a71ffb640 1 -- 172.21.15.112:0/3195030107 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a8004ae50 con 0x555852ec31b0 2024-04-25T16:38:44.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.010+0000 7f3a71ffb640 1 -- 172.21.15.112:0/3195030107 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80004990 con 0x555852ec31b0 2024-04-25T16:38:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.010+0000 7f3a991af740 1 -- 172.21.15.112:0/3195030107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 msgr2=0x55585326ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:44.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.010+0000 7f3a991af740 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 0x55585326ad30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3a800096c0 tx=0x7f3a80031400 comp rx=0 tx=0).stop 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- 172.21.15.112:0/3195030107 shutdown_connections 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852ea24d0 0x555853270310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ec31b0 0x55585326ad30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 --2- 172.21.15.112:0/3195030107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852e9a2f0 0x555853277440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- 172.21.15.112:0/3195030107 >> 172.21.15.112:0/3195030107 conn(0x555852eb7300 msgr2=0x55585326fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- 172.21.15.112:0/3195030107 shutdown_connections 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- 172.21.15.112:0/3195030107 wait complete. 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- start start 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 Processor -- start 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x5558532791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853279710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x5558532e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1ac010 con 0x555852ea24d0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1ad480 con 0x555852ec31b0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1ac6e0 con 0x555852e9a2f0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.011+0000 7f3a991af740 10 client.0 _ll_register_callbacks cb 0x7f3a80009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853279710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853279710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41556/0 (socket says 172.21.15.112:41556) 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 -- 172.21.15.112:0/4104605426 learned_addr learned my addr 172.21.15.112:0/4104605426 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x5558532e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x5558532791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 -- 172.21.15.112:0/4104605426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 msgr2=0x5558532791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:44.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x5558532791d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 -- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 msgr2=0x5558532e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x5558532e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 -- 172.21.15.112:0/4104605426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558532e4000 con 0x555852ea24d0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x5558532791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a8bfff640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853279710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c024600 tx=0x7f3a7c024930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1391b675b87d75d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/4104605426 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c009a20 con 0x555852ea24d0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/4104605426 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f3a7c003d30 con 0x555852ea24d0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8004ae50 con 0x555852ea24d0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/4104605426 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c044050 con 0x555852ea24d0 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a991af740 10 client.15411 fetch_fsmap learned FSMap version 22 2024-04-25T16:38:44.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.012+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5558532eb2f0 con 0x555852ea24d0 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/4104605426 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a8004ae50 con 0x555852ea24d0 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/4104605426 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f3a7c0035f0 con 0x555852ea24d0 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a991af740 10 client.15411 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a991af740 10 client.15411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a991af740 20 client.15411 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a991af740 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e571c0 0x5558532eda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a991af740 4 client.15411 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558531d0ee0 con 0x555852e571c0 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e571c0 0x5558532eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a6b7fe640 10 client.15411 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:44.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.013+0000 7f3a8b7fe640 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e571c0 0x5558532eda40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1798d8da092696df server_cookie=ab8ee49136470214 in_seq=0 out_seq=0 2024-04-25T16:38:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.025+0000 7f3a6b7fe640 1 -- 172.21.15.112:0/4104605426 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558531d0ee0 con 0x555852e571c0 2024-04-25T16:38:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.025+0000 7f3a6b7fe640 10 client.15411 handle_command_reply: tid=0 2024-04-25T16:38:44.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.025+0000 7f3a991af740 1 client.15411 shutdown 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.025+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 msgr2=0x555853279710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853279710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c024600 tx=0x7f3a7c024930 comp rx=0 tx=0).stop 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 shutdown_connections 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555852ec31b0 0x5558532e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555852ea24d0 0x555853279710 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555852e571c0 0x5558532eda40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 --2- 172.21.15.112:0/4104605426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555852e9a2f0 0x5558532791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a6b7fe640 0 client.15411 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 >> 172.21.15.112:0/4104605426 conn(0x555852eb7300 msgr2=0x555853262880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 shutdown_connections 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 1 -- 172.21.15.112:0/4104605426 wait complete. 2024-04-25T16:38:44.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a991af740 20 client.15411 trim_cache size 0 max 16384 2024-04-25T16:38:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:38:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:38:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:38:44.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3a740afd70 msgr2=0x7f3a740b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3a740afd70 0x7f3a740b2280 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3a840dcbd0 tx=0x7f3a84074d50 comp rx=0 tx=0).stop 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 msgr2=0x7f3a8c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c19ef80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400eaa0 tx=0x7f3a8400ef60 comp rx=0 tx=0).stop 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 shutdown_connections 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3a740afd70 0x7f3a740b2280 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3a8c10a000 0x7f3a8c19ef80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a8c109b60 0x7f3a8c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 --2- 172.21.15.112:0/3583144241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a8c1042b0 0x7f3a8c19a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 >> 172.21.15.112:0/3583144241 conn(0x7f3a8c079170 msgr2=0x7f3a8c1022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.026+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 shutdown_connections 2024-04-25T16:38:44.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:44.027+0000 7f3a922f9640 1 -- 172.21.15.112:0/3583144241 wait complete. 2024-04-25T16:38:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:45 smithi116 ceph-mon[32712]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 77 KiB/s wr, 24 op/s 2024-04-25T16:38:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:45 smithi112 ceph-mon[29122]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 77 KiB/s wr, 24 op/s 2024-04-25T16:38:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:45 smithi133 ceph-mon[32691]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 77 KiB/s wr, 24 op/s 2024-04-25T16:38:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:47 smithi116 ceph-mon[32712]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-04-25T16:38:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:47 smithi112 ceph-mon[29122]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-04-25T16:38:47.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:47 smithi133 ceph-mon[32691]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-04-25T16:38:49.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:49 smithi116 ceph-mon[32712]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:38:49.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:49 smithi112 ceph-mon[29122]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:38:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:49 smithi133 ceph-mon[32691]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:38:51.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:51 smithi116 ceph-mon[32712]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T16:38:51.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:51 smithi112 ceph-mon[29122]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T16:38:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:51 smithi133 ceph-mon[32691]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T16:38:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:53 smithi116 ceph-mon[32712]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:38:53.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:53 smithi112 ceph-mon[29122]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:38:53.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:53 smithi133 ceph-mon[32691]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:38:55.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:55 smithi112 ceph-mon[29122]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-25T16:38:55.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:55 smithi133 ceph-mon[32691]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-25T16:38:55.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:55 smithi116 ceph-mon[32712]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3303233162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108a60 msgr2=0x7f51e8102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3303233162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108a60 0x7f51e8102610 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009900 tx=0x7f51d802a410 comp rx=0 tx=0).stop 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3303233162 shutdown_connections 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3303233162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3303233162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51e8108f00 0x7f51e8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3303233162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108a60 0x7f51e8102610 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3303233162 >> 172.21.15.112:0/3303233162 conn(0x7f51e80781b0 msgr2=0x7f51e80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3303233162 shutdown_connections 2024-04-25T16:38:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.134+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3303233162 wait complete. 2024-04-25T16:38:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 Processor -- start 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 -- start start 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51e8108a60 0x7f51e8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108f00 0x7f51e819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8112c60 con 0x7f51e8108f00 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8112ae0 con 0x7f51e8108a60 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.135+0000 7f51ed5a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51e8112de0 con 0x7f51e81031c0 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108f00 0x7f51e819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e8199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:47004/0 (socket says 172.21.15.112:47004) 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 -- 172.21.15.112:0/3416785056 learned_addr learned my addr 172.21.15.112:0/3416785056 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e67fc640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51e8108a60 0x7f51e8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 -- 172.21.15.112:0/3416785056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51e8108a60 msgr2=0x7f51e8199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51e8108a60 0x7f51e8199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 -- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108f00 msgr2=0x7f51e819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108f00 0x7f51e819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 -- 172.21.15.112:0/3416785056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51e819e420 con 0x7f51e81031c0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e8199640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51d802a930 tx=0x7f51d8004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51cffff640 1 -- 172.21.15.112:0/3416785056 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d803f030 con 0x7f51e81031c0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51e819e6b0 con 0x7f51e81031c0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51cffff640 1 -- 172.21.15.112:0/3416785056 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51d8003aa0 con 0x7f51e81031c0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51cffff640 1 -- 172.21.15.112:0/3416785056 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8035070 con 0x7f51e81031c0 2024-04-25T16:38:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.136+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51e819eba0 con 0x7f51e81031c0 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51cffff640 1 -- 172.21.15.112:0/3416785056 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 23) ==== 147748+0+0 (secure 0 0 0) 0x7f51d8042030 con 0x7f51e81031c0 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 Processor -- start 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 -- start start 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 0x560505dc7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dc8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505dc8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560505d2e460 con 0x560505a281b0 2024-04-25T16:38:57.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560505ce8590 con 0x5605059ff2f0 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51f4456740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560505d2a420 con 0x560505a074d0 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 0x560505dc7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51e77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 0x560505dc7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43998/0 (socket says 172.21.15.112:43998) 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51e77fe640 1 -- 172.21.15.112:0/2081490470 learned_addr learned my addr 172.21.15.112:0/2081490470 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51e67fc640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505dc8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.139+0000 7f51cffff640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51d00afd70 0x7f51d00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dc8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 -- 172.21.15.112:0/2081490470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 msgr2=0x560505dc8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505dc8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 -- 172.21.15.112:0/2081490470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 msgr2=0x560505dc8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dc8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 -- 172.21.15.112:0/2081490470 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560505cf50d0 con 0x560505a281b0 2024-04-25T16:38:57.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e67fc640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505dc8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dc8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51d00afd70 0x7f51d00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51cffff640 1 -- 172.21.15.112:0/3416785056 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f51d80ff680 con 0x7f51e81031c0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51e77fe640 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 0x560505dc7af0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f51e0011590 tx=0x7f51e0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad27f0937da9c3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51cd7fa640 1 -- 172.21.15.112:0/2081490470 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e0017050 con 0x560505a281b0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51cd7fa640 1 -- 172.21.15.112:0/2081490470 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e000fcf0 con 0x560505a281b0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.140+0000 7f51cd7fa640 1 -- 172.21.15.112:0/2081490470 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e001b940 con 0x560505a281b0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 -- 172.21.15.112:0/2081490470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 msgr2=0x560505dc7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51e77fe640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51d00afd70 0x7f51d00b2280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f51e0022000 tx=0x7f51e0023000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 0x560505dc7af0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f51e0011590 tx=0x7f51e0011a50 comp rx=0 tx=0).stop 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 -- 172.21.15.112:0/2081490470 shutdown_connections 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505dc8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dc8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 --2- 172.21.15.112:0/2081490470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a281b0 0x560505dc7af0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 -- 172.21.15.112:0/2081490470 >> 172.21.15.112:0/2081490470 conn(0x560505a1c300 msgr2=0x560505dc42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 -- 172.21.15.112:0/2081490470 shutdown_connections 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 -- 172.21.15.112:0/2081490470 wait complete. 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 -- start start 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.141+0000 7f51f4456740 1 Processor -- start 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505dd21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a074d0 0x560505dd2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dd8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560505dd4c20 con 0x5605059ff2f0 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560505cb5b60 con 0x560505a074d0 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560505dd4da0 con 0x560505a281b0 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51f4456740 10 client.0 _ll_register_callbacks cb 0x560505cda2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505dd21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505dd21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44010/0 (socket says 172.21.15.112:44010) 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 -- 172.21.15.112:0/1495956891 learned_addr learned my addr 172.21.15.112:0/1495956891 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e77fe640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dd8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e67fc640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a074d0 0x560505dd2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 -- 172.21.15.112:0/1495956891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 msgr2=0x560505dd8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dd8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 -- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a074d0 msgr2=0x560505dd2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a074d0 0x560505dd2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 -- 172.21.15.112:0/1495956891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560505a40210 con 0x5605059ff2f0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505dd21f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f51d80fcd40 tx=0x7f51d80c2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2b0e4fa2c6d9b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.142+0000 7f51caff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51caff5640 1 -- 172.21.15.112:0/1495956891 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80c6ea0 con 0x5605059ff2f0 2024-04-25T16:38:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e000fcf0 con 0x5605059ff2f0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51caff5640 1 -- 172.21.15.112:0/1495956891 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f51d80c39c0 con 0x5605059ff2f0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51caff5640 1 -- 172.21.15.112:0/1495956891 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f51d80c7240 con 0x5605059ff2f0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 10 client.15423 fetch_fsmap learned FSMap version 22 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560505b0c620 con 0x5605059ff2f0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51caff5640 1 -- 172.21.15.112:0/1495956891 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e000fcf0 con 0x5605059ff2f0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51caff5640 1 -- 172.21.15.112:0/1495956891 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f51d80c7990 con 0x5605059ff2f0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 10 client.15423 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 20 client.15423 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059e6d80 0x560505de4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 4 client.15423 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.143+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560505d3c070 con 0x5605059e6d80 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.144+0000 7f51e67fc640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059e6d80 0x560505de4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.144+0000 7f51e67fc640 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059e6d80 0x560505de4660 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8eeae5bb83a978b server_cookie=b5d837adb500ea04 in_seq=0 out_seq=0 2024-04-25T16:38:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.144+0000 7f51caff5640 10 client.15423 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51caff5640 1 -- 172.21.15.112:0/1495956891 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560505d3c070 con 0x5605059e6d80 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51caff5640 10 client.15423 handle_command_reply: tid=0 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 client.15423 shutdown 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 msgr2=0x560505dd21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505dd21f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f51d80fcd40 tx=0x7f51d80c2f90 comp rx=0 tx=0).stop 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 shutdown_connections 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059e6d80 0x560505de4660 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dd8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a074d0 0x560505dd2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51caff5640 0 client.15423 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 --2- 172.21.15.112:0/1495956891 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505dd21f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 >> 172.21.15.112:0/1495956891 conn(0x560505a1c300 msgr2=0x560505dc7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 shutdown_connections 2024-04-25T16:38:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 1 -- 172.21.15.112:0/1495956891 wait complete. 2024-04-25T16:38:57.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.145+0000 7f51f4456740 20 client.15423 trim_cache size 0 max 16384 2024-04-25T16:38:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 Processor -- start 2024-04-25T16:38:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 -- start start 2024-04-25T16:38:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dcbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 0x560505de9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.163 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81b11e0 con 0x560505a074d0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81aeab0 con 0x5605059ff2f0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51f4456740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81ac900 con 0x560505a281b0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51e67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 0x560505de9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51e67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 0x560505de9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44012/0 (socket says 172.21.15.112:44012) 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51e67fc640 1 -- 172.21.15.112:0/745432884 learned_addr learned my addr 172.21.15.112:0/745432884 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51e77fe640 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dcbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.160+0000 7f51e67fc640 1 -- 172.21.15.112:0/745432884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 msgr2=0x560505dea3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51e67fc640 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dea3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51e67fc640 1 -- 172.21.15.112:0/745432884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 msgr2=0x560505dcbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51e67fc640 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dcbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51e67fc640 1 -- 172.21.15.112:0/745432884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560505dbf710 con 0x560505a074d0 2024-04-25T16:38:57.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dcbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51e67fc640 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 0x560505de9e60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00a5c0 tx=0x7f51dc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38612798049fac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51ccff9640 1 -- 172.21.15.112:0/745432884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc004220 con 0x560505a074d0 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51ccff9640 1 -- 172.21.15.112:0/745432884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51dc024030 con 0x560505a074d0 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.161+0000 7f51ccff9640 1 -- 172.21.15.112:0/745432884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0203b0 con 0x560505a074d0 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- 172.21.15.112:0/745432884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 msgr2=0x560505de9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 0x560505de9e60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00a5c0 tx=0x7f51dc005f90 comp rx=0 tx=0).stop 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- 172.21.15.112:0/745432884 shutdown_connections 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a281b0 0x560505dea3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560505a074d0 0x560505de9e60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- 172.21.15.112:0/745432884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605059ff2f0 0x560505dcbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- 172.21.15.112:0/745432884 >> 172.21.15.112:0/745432884 conn(0x560505a1c300 msgr2=0x560505dcb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- 172.21.15.112:0/745432884 shutdown_connections 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- 172.21.15.112:0/745432884 wait complete. 2024-04-25T16:38:57.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- start start 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 Processor -- start 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505e429c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505e42f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a281b0 0x560505e43440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81acf70 con 0x5605059ff2f0 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81ae3e0 con 0x560505a281b0 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51e81ad640 con 0x560505a074d0 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51f4456740 10 client.0 _ll_register_callbacks cb 0x7f51d8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505e429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51e6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505e429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:44024/0 (socket says 172.21.15.112:44024) 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51e6ffd640 1 -- 172.21.15.112:0/4248543393 learned_addr learned my addr 172.21.15.112:0/4248543393 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.162+0000 7f51e67fc640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505e42f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e77fe640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a281b0 0x560505e43440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e6ffd640 1 -- 172.21.15.112:0/4248543393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 msgr2=0x560505e42f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505e42f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e6ffd640 1 -- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a281b0 msgr2=0x560505e43440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a281b0 0x560505e43440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e6ffd640 1 -- 172.21.15.112:0/4248543393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560505e4db70 con 0x5605059ff2f0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e77fe640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a281b0 0x560505e43440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51e6ffd640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505e429c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f51d80c73e0 tx=0x7f51d803b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a2ee9f4334ed60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51caff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51caff5640 1 -- 172.21.15.112:0/4248543393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80e6060 con 0x5605059ff2f0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51caff5640 1 -- 172.21.15.112:0/4248543393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f51d80d0050 con 0x5605059ff2f0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51dc024030 con 0x5605059ff2f0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51caff5640 1 -- 172.21.15.112:0/4248543393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f51d80a7050 con 0x5605059ff2f0 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51f4456740 10 client.15429 fetch_fsmap learned FSMap version 22 2024-04-25T16:38:57.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x560505e4ed70 con 0x5605059ff2f0 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.163+0000 7f51caff5640 1 -- 172.21.15.112:0/4248543393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51dc024030 con 0x5605059ff2f0 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51caff5640 1 -- 172.21.15.112:0/4248543393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f51d80fcf10 con 0x5605059ff2f0 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51f4456740 10 client.15429 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51f4456740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51f4456740 20 client.15429 populate_metadata read hostname 'smithi112' 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51f4456740 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059bc1c0 0x560505e513c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51f4456740 4 client.15429 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560505d35ee0 con 0x5605059bc1c0 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51e67fc640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059bc1c0 0x560505e513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51caff5640 10 client.15429 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:57.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.164+0000 7f51e67fc640 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059bc1c0 0x560505e513c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283a9be21eb661e4 server_cookie=39900572d66c2a96 in_seq=0 out_seq=0 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51caff5640 1 -- 172.21.15.112:0/4248543393 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560505d35ee0 con 0x5605059bc1c0 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51caff5640 10 client.15429 handle_command_reply: tid=0 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 client.15429 shutdown 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 msgr2=0x560505e429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505e429c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f51d80c73e0 tx=0x7f51d803b5d0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 shutdown_connections 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605059bc1c0 0x560505e513c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560505a074d0 0x560505e42f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560505a281b0 0x560505e43440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51caff5640 0 client.15429 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 --2- 172.21.15.112:0/4248543393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605059ff2f0 0x560505e429c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.170 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 >> 172.21.15.112:0/4248543393 conn(0x560505a1c300 msgr2=0x560505dc7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 shutdown_connections 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 1 -- 172.21.15.112:0/4248543393 wait complete. 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51f4456740 20 client.15429 trim_cache size 0 max 16384 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:38:57.171 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51d00afd70 msgr2=0x7f51d00b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51d00afd70 0x7f51d00b2280 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f51e0022000 tx=0x7f51e0023000 comp rx=0 tx=0).stop 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 msgr2=0x7f51e8199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.168+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e8199640 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f51d802a930 tx=0x7f51d8004060 comp rx=0 tx=0).stop 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 shutdown_connections 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51d00afd70 0x7f51d00b2280 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.172 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51e8108f00 0x7f51e819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51e8108a60 0x7f51e8199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 --2- 172.21.15.112:0/3416785056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51e81031c0 0x7f51e8199640 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:38:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 >> 172.21.15.112:0/3416785056 conn(0x7f51e80781b0 msgr2=0x7f51e8104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:38:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 shutdown_connections 2024-04-25T16:38:57.173 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:38:57.169+0000 7f51ed5a8640 1 -- 172.21.15.112:0/3416785056 wait complete. 2024-04-25T16:38:57.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:57 smithi112 ceph-mon[29122]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T16:38:57.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:57 smithi133 ceph-mon[32691]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T16:38:57.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:57 smithi116 ceph-mon[32712]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-04-25T16:38:59.560 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:38:59 smithi116 ceph-mon[32712]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:38:59.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:38:59 smithi112 ceph-mon[29122]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:38:59.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:38:59 smithi133 ceph-mon[32691]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:39:00.345 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:39:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:39:00.035+0000 7fb4fde1a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:39:00.345 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:39:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:39:00.035+0000 7fb4fde1a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:39:00.345 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:39:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:39:00.035+0000 7fb4fde1a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:39:00.345 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:39:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:39:00.035+0000 7fb4fde1a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:39:00.346 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:39:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:39:00.035+0000 7fb4fde1a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T16:39:01.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:01 smithi112 ceph-mon[29122]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-25T16:39:01.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:01 smithi133 ceph-mon[32691]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-25T16:39:01.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:01 smithi116 ceph-mon[32712]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-25T16:39:03.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:03 smithi112 ceph-mon[29122]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:03.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:03 smithi112 ceph-mon[29122]: mgrmap e24: x(active, since 34s), standbys: z, y 2024-04-25T16:39:03.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:03 smithi133 ceph-mon[32691]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:03.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:03 smithi133 ceph-mon[32691]: mgrmap e24: x(active, since 34s), standbys: z, y 2024-04-25T16:39:03.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:03 smithi116 ceph-mon[32712]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:03.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:03 smithi116 ceph-mon[32712]: mgrmap e24: x(active, since 34s), standbys: z, y 2024-04-25T16:39:05.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:05 smithi112 ceph-mon[29122]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-04-25T16:39:05.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:05 smithi133 ceph-mon[32691]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-04-25T16:39:05.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:05 smithi116 ceph-mon[32712]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-04-25T16:39:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:07 smithi112 ceph-mon[29122]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T16:39:07.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:07 smithi133 ceph-mon[32691]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T16:39:07.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:07 smithi116 ceph-mon[32712]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 23 KiB/s wr, 12 op/s 2024-04-25T16:39:09.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:09 smithi112 ceph-mon[29122]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T16:39:09.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:09 smithi133 ceph-mon[32691]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T16:39:09.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:09 smithi116 ceph-mon[32712]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 -- 172.21.15.112:0/3808873375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 msgr2=0x7f6774102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 --2- 172.21.15.112:0/3808873375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f6774102b70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f676802a410 comp rx=0 tx=0).stop 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 -- 172.21.15.112:0/3808873375 shutdown_connections 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 --2- 172.21.15.112:0/3808873375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 0x7f677410f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 --2- 172.21.15.112:0/3808873375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f6774102b70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 --2- 172.21.15.112:0/3808873375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 0x7f6774102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 -- 172.21.15.112:0/3808873375 >> 172.21.15.112:0/3808873375 conn(0x7f6774078080 msgr2=0x7f67740784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 -- 172.21.15.112:0/3808873375 shutdown_connections 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 -- 172.21.15.112:0/3808873375 wait complete. 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 Processor -- start 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.272+0000 7f677a8b6640 1 -- start start 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f677a8b6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 0x7f67741996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f677a8b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 0x7f6774199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f677a8b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f677419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f677a8b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112c80 con 0x7f6774108f30 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f677a8b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112b00 con 0x7f6774108a90 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f677a8b6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6774112e00 con 0x7f67741031e0 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f677419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f677419dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45976/0 (socket says 172.21.15.112:45976) 2024-04-25T16:39:10.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 -- 172.21.15.112:0/836181485 learned_addr learned my addr 172.21.15.112:0/836181485 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f67737fe640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 0x7f6774199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6773fff640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 0x7f67741996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 -- 172.21.15.112:0/836181485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 msgr2=0x7f67741996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 0x7f67741996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 -- 172.21.15.112:0/836181485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 msgr2=0x7f6774199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 0x7f6774199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6778e53640 1 -- 172.21.15.112:0/836181485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419e520 con 0x7f6774108f30 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f67737fe640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 0x7f6774199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.273+0000 7f6773fff640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 0x7f67741996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.274+0000 7f6778e53640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f677419dee0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f676c009fd0 tx=0x7f676c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.274+0000 7f67717fa640 1 -- 172.21.15.112:0/836181485 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c017070 con 0x7f6774108f30 2024-04-25T16:39:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.274+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419e810 con 0x7f6774108f30 2024-04-25T16:39:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.274+0000 7f67717fa640 1 -- 172.21.15.112:0/836181485 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676c003e20 con 0x7f6774108f30 2024-04-25T16:39:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.274+0000 7f67717fa640 1 -- 172.21.15.112:0/836181485 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c010460 con 0x7f6774108f30 2024-04-25T16:39:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.274+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67741a6190 con 0x7f6774108f30 2024-04-25T16:39:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.276+0000 7f67717fa640 1 -- 172.21.15.112:0/836181485 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 147805+0+0 (secure 0 0 0) 0x7f676c003970 con 0x7f6774108f30 2024-04-25T16:39:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.276+0000 7f678176e740 1 Processor -- start 2024-04-25T16:39:10.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 -- start start 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb06faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 0x5581cb070570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581cafd6460 con 0x5581cacaf4d0 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581caf90590 con 0x5581cacd01b0 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f678176e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5581cafd2420 con 0x5581caca72f0 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67717fa640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f675c0afe00 0x7f675c0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 0x5581cb070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 0x5581cb070570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45978/0 (socket says 172.21.15.112:45978) 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 -- 172.21.15.112:0/2050648266 learned_addr learned my addr 172.21.15.112:0/2050648266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f6778e53640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f675c0afe00 0x7f675c0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f6773fff640 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb06faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67717fa640 1 -- 172.21.15.112:0/836181485 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f676c0df110 con 0x7f6774108f30 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 -- 172.21.15.112:0/2050648266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 msgr2=0x5581cb070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 -- 172.21.15.112:0/2050648266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 msgr2=0x5581cb06faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb06faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 -- 172.21.15.112:0/2050648266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581caf9d0d0 con 0x5581cacaf4d0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.277+0000 7f67737fe640 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 0x5581cb070570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f6768000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e7c3b5e42674c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f6778e53640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f675c0afe00 0x7f675c0b2310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f676c0dc700 tx=0x7f676c075bf0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f675a7fc640 1 -- 172.21.15.112:0/2050648266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768035040 con 0x5581cacaf4d0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f675a7fc640 1 -- 172.21.15.112:0/2050648266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6768004030 con 0x5581cacaf4d0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f675a7fc640 1 -- 172.21.15.112:0/2050648266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768003a50 con 0x5581cacaf4d0 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f678176e740 1 -- 172.21.15.112:0/2050648266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 msgr2=0x5581cb070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f678176e740 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 0x5581cb070570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f6768000bc0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f678176e740 1 -- 172.21.15.112:0/2050648266 shutdown_connections 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f678176e740 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f678176e740 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacaf4d0 0x5581cb070570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.278+0000 7f678176e740 1 --2- 172.21.15.112:0/2050648266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb06faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- 172.21.15.112:0/2050648266 >> 172.21.15.112:0/2050648266 conn(0x5581cacc4300 msgr2=0x5581cb06c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- 172.21.15.112:0/2050648266 shutdown_connections 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- 172.21.15.112:0/2050648266 wait complete. 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- start start 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 Processor -- start 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581caca72f0 0x5581cb07bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb07c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb07a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.286 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5581cb07cbc0 con 0x5581cacd01b0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581caf5db60 con 0x5581caca72f0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5581cb07cd40 con 0x5581cacaf4d0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.279+0000 7f678176e740 10 client.0 _ll_register_callbacks cb 0x5581caf822b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb07a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb07a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45980/0 (socket says 172.21.15.112:45980) 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 -- 172.21.15.112:0/2637980852 learned_addr learned my addr 172.21.15.112:0/2637980852 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f6773fff640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb07c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581caca72f0 0x5581cb07bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 -- 172.21.15.112:0/2637980852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 msgr2=0x5581cb07c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb07c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.287 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 -- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581caca72f0 msgr2=0x5581cb07bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581caca72f0 0x5581cb07bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 -- 172.21.15.112:0/2637980852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581cace8210 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67737fe640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb07a050 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6768001eb0 tx=0x7f6768003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7faa27bcfec2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67537fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67537fe640 1 -- 172.21.15.112:0/2637980852 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768003810 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6768004030 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67537fe640 1 -- 172.21.15.112:0/2637980852 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f6768044030 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f67537fe640 1 -- 172.21.15.112:0/2637980852 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6768059050 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f678176e740 10 client.15447 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.280+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5581cadb4620 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f67537fe640 1 -- 172.21.15.112:0/2637980852 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6768004030 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f67537fe640 1 -- 172.21.15.112:0/2637980852 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f676803a040 con 0x5581cacd01b0 2024-04-25T16:39:10.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f678176e740 10 client.15447 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f678176e740 10 client.15447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f678176e740 20 client.15447 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f678176e740 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac8ed80 0x5581cb08c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f678176e740 4 client.15447 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581cafe4070 con 0x5581cac8ed80 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.281+0000 7f6773fff640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac8ed80 0x5581cb08c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.282+0000 7f67537fe640 10 client.15447 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.282+0000 7f6773fff640 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac8ed80 0x5581cb08c5a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0175de8fbd2743b server_cookie=6c3fcf8971a71187 in_seq=0 out_seq=0 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.282+0000 7f67537fe640 1 -- 172.21.15.112:0/2637980852 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5581cafe4070 con 0x5581cac8ed80 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.282+0000 7f67537fe640 10 client.15447 handle_command_reply: tid=0 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.282+0000 7f678176e740 1 client.15447 shutdown 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 msgr2=0x5581cb07a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb07a050 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6768001eb0 tx=0x7f6768003730 comp rx=0 tx=0).stop 2024-04-25T16:39:10.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 shutdown_connections 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac8ed80 0x5581cb08c5a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb07c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb07a050 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f67537fe640 0 client.15447 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 --2- 172.21.15.112:0/2637980852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581caca72f0 0x5581cb07bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 >> 172.21.15.112:0/2637980852 conn(0x5581cacc4300 msgr2=0x5581cb071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 shutdown_connections 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 1 -- 172.21.15.112:0/2637980852 wait complete. 2024-04-25T16:39:10.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.283+0000 7f678176e740 20 client.15447 trim_cache size 0 max 16384 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.298+0000 7f678176e740 1 Processor -- start 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.298+0000 7f678176e740 1 -- start start 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.298+0000 7f678176e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 0x5581cb070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.298+0000 7f678176e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb08f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.298+0000 7f678176e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb08fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f678176e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67741b11b0 con 0x5581caca72f0 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f678176e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67741aea80 con 0x5581cacd01b0 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f678176e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ac8d0 con 0x5581cacaf4d0 2024-04-25T16:39:10.302 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 0x5581cb070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 0x5581cb070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45988/0 (socket says 172.21.15.112:45988) 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 -- 172.21.15.112:0/2857336545 learned_addr learned my addr 172.21.15.112:0/2857336545 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f6773fff640 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb08f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 -- 172.21.15.112:0/2857336545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 msgr2=0x5581cb08f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb08f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 -- 172.21.15.112:0/2857336545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 msgr2=0x5581cb08fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb08fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 -- 172.21.15.112:0/2857336545 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581cb090050 con 0x5581caca72f0 2024-04-25T16:39:10.303 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f6773fff640 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb08f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f67737fe640 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 0x5581cb070180 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6768057040 tx=0x7f67680095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dea32ca3db21770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f6759ffb640 1 -- 172.21.15.112:0/2857336545 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768035070 con 0x5581caca72f0 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f6759ffb640 1 -- 172.21.15.112:0/2857336545 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676803d900 con 0x5581caca72f0 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.299+0000 7f6759ffb640 1 -- 172.21.15.112:0/2857336545 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67680037a0 con 0x5581caca72f0 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 -- 172.21.15.112:0/2857336545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 msgr2=0x5581cb070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 0x5581cb070180 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6768057040 tx=0x7f67680095f0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 -- 172.21.15.112:0/2857336545 shutdown_connections 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581cacaf4d0 0x5581cb08f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacd01b0 0x5581cb08fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 --2- 172.21.15.112:0/2857336545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581caca72f0 0x5581cb070180 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 -- 172.21.15.112:0/2857336545 >> 172.21.15.112:0/2857336545 conn(0x5581cacc4300 msgr2=0x5581cb06fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 -- 172.21.15.112:0/2857336545 shutdown_connections 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.300+0000 7f678176e740 1 -- 172.21.15.112:0/2857336545 wait complete. 2024-04-25T16:39:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 -- start start 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 Processor -- start 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb0ecd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacaf4d0 0x5581cb0ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb08ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f67741acf40 con 0x5581cacd01b0 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ae3b0 con 0x5581cacaf4d0 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67741ad610 con 0x5581caca72f0 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f678176e740 10 client.0 _ll_register_callbacks cb 0x7f6768032dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb08ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb08ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45992/0 (socket says 172.21.15.112:45992) 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 -- 172.21.15.112:0/2942671564 learned_addr learned my addr 172.21.15.112:0/2942671564 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f6773fff640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacaf4d0 0x5581cb0ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.305 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb0ecd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 -- 172.21.15.112:0/2942671564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 msgr2=0x5581cb0ecd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb0ecd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.301+0000 7f67737fe640 1 -- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacaf4d0 msgr2=0x5581cb0ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67737fe640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacaf4d0 0x5581cb0ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67737fe640 1 -- 172.21.15.112:0/2942671564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581cb08f080 con 0x5581cacd01b0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67737fe640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb08ea40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f676803d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a3b76d5031c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67537fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67537fe640 1 -- 172.21.15.112:0/2942671564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768035070 con 0x5581cacd01b0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67537fe640 1 -- 172.21.15.112:0/2942671564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f67680039e0 con 0x5581cacd01b0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f676803d900 con 0x5581cacd01b0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67537fe640 1 -- 172.21.15.112:0/2942671564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f6768005250 con 0x5581cacd01b0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 10 client.15453 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5581cb0f4320 con 0x5581cacd01b0 2024-04-25T16:39:10.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67537fe640 1 -- 172.21.15.112:0/2942671564 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f676803d900 con 0x5581cacd01b0 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f67537fe640 1 -- 172.21.15.112:0/2942671564 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f67680314a0 con 0x5581cacd01b0 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 10 client.15453 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 10 client.15453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 20 client.15453 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac641c0 0x5581cb0f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 4 client.15453 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.302+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581cafddee0 con 0x5581cac641c0 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.303+0000 7f6773fff640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac641c0 0x5581cb0f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.303+0000 7f67537fe640 10 client.15453 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.303+0000 7f6773fff640 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac641c0 0x5581cb0f6990 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab0d42753adf75c server_cookie=d0682eb70e088c29 in_seq=0 out_seq=0 2024-04-25T16:39:10.310 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.311+0000 7f67537fe640 1 -- 172.21.15.112:0/2942671564 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581cafddee0 con 0x5581cac641c0 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.311+0000 7f67537fe640 10 client.15453 handle_command_reply: tid=0 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.311+0000 7f678176e740 1 client.15453 shutdown 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 msgr2=0x5581cb08ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb08ea40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f676803d550 comp rx=0 tx=0).stop 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 shutdown_connections 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5581cacd01b0 0x5581cb08ea40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581cacaf4d0 0x5581cb0ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5581cac641c0 0x5581cb0f6990 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 --2- 172.21.15.112:0/2942671564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5581caca72f0 0x5581cb0ecd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f67537fe640 0 client.15453 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 >> 172.21.15.112:0/2942671564 conn(0x5581cacc4300 msgr2=0x5581cb06f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 shutdown_connections 2024-04-25T16:39:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 1 -- 172.21.15.112:0/2942671564 wait complete. 2024-04-25T16:39:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f678176e740 20 client.15453 trim_cache size 0 max 16384 2024-04-25T16:39:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:39:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:39:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:39:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f675c0afe00 msgr2=0x7f675c0b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f677a8b6640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f675c0afe00 0x7f675c0b2310 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f676c0dc700 tx=0x7f676c075bf0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 msgr2=0x7f677419dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.312+0000 7f677a8b6640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f677419dee0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f676c009fd0 tx=0x7f676c00b260 comp rx=0 tx=0).stop 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 shutdown_connections 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f675c0afe00 0x7f675c0b2310 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6774108f30 0x7f677419dee0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6774108a90 0x7f6774199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 --2- 172.21.15.112:0/836181485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67741031e0 0x7f67741996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 >> 172.21.15.112:0/836181485 conn(0x7f6774078080 msgr2=0x7f67741011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 shutdown_connections 2024-04-25T16:39:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:10.313+0000 7f677a8b6640 1 -- 172.21.15.112:0/836181485 wait complete. 2024-04-25T16:39:11.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:11 smithi112 ceph-mon[29122]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T16:39:11.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:11 smithi133 ceph-mon[32691]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T16:39:11.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:11 smithi116 ceph-mon[32712]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-25T16:39:13.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:13 smithi112 ceph-mon[29122]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-04-25T16:39:13.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:13 smithi133 ceph-mon[32691]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-04-25T16:39:13.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:13 smithi116 ceph-mon[32712]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-04-25T16:39:15.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:15 smithi112 ceph-mon[29122]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T16:39:15.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:15 smithi133 ceph-mon[32691]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T16:39:15.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:15 smithi116 ceph-mon[32712]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 25 KiB/s wr, 15 op/s 2024-04-25T16:39:17.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:17 smithi112 ceph-mon[29122]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-25T16:39:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:17 smithi133 ceph-mon[32691]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-25T16:39:17.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:17 smithi116 ceph-mon[32712]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-25T16:39:19.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:19 smithi112 ceph-mon[29122]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T16:39:19.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:19 smithi133 ceph-mon[32691]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T16:39:19.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:19 smithi116 ceph-mon[32712]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T16:39:21.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:21 smithi112 ceph-mon[29122]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-25T16:39:21.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:21 smithi133 ceph-mon[32691]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-25T16:39:21.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:21 smithi116 ceph-mon[32712]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-25T16:39:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 -- 172.21.15.112:0/3484925819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 msgr2=0x7f40b010f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 --2- 172.21.15.112:0/3484925819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b010f4f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009ca0 tx=0x7f40a002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 -- 172.21.15.112:0/3484925819 shutdown_connections 2024-04-25T16:39:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 --2- 172.21.15.112:0/3484925819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b010f4f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 --2- 172.21.15.112:0/3484925819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40b0108f00 0x7f40b0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 --2- 172.21.15.112:0/3484925819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b0108a60 0x7f40b0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 -- 172.21.15.112:0/3484925819 >> 172.21.15.112:0/3484925819 conn(0x7f40b00781b0 msgr2=0x7f40b00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 -- 172.21.15.112:0/3484925819 shutdown_connections 2024-04-25T16:39:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.434+0000 7f40b52c6640 1 -- 172.21.15.112:0/3484925819 wait complete. 2024-04-25T16:39:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 Processor -- start 2024-04-25T16:39:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 -- start start 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40b0108a60 0x7f40b0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b0108f00 0x7f40b019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0112cd0 con 0x7f40b01031c0 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0112b50 con 0x7f40b0108a60 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40b52c6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0112e50 con 0x7f40b0108f00 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b0199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51330/0 (socket says 172.21.15.112:51330) 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.435+0000 7f40aeffd640 1 -- 172.21.15.112:0/52546963 learned_addr learned my addr 172.21.15.112:0/52546963 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40af7fe640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b0108f00 0x7f40b019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40ae7fc640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40b0108a60 0x7f40b0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40aeffd640 1 -- 172.21.15.112:0/52546963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b0108f00 msgr2=0x7f40b019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40aeffd640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b0108f00 0x7f40b019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40aeffd640 1 -- 172.21.15.112:0/52546963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40b0108a60 msgr2=0x7f40b0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40aeffd640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40b0108a60 0x7f40b0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40aeffd640 1 -- 172.21.15.112:0/52546963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b019e410 con 0x7f40b01031c0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40aeffd640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b0199630 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c8f0 tx=0x7f40a800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f4097fff640 1 -- 172.21.15.112:0/52546963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8007bb0 con 0x7f40b01031c0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b019e700 con 0x7f40b01031c0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f4097fff640 1 -- 172.21.15.112:0/52546963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8007d50 con 0x7f40b01031c0 2024-04-25T16:39:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f4097fff640 1 -- 172.21.15.112:0/52546963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8018780 con 0x7f40b01031c0 2024-04-25T16:39:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.436+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b006a3c0 con 0x7f40b01031c0 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f4097fff640 1 -- 172.21.15.112:0/52546963 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 147805+0+0 (secure 0 0 0) 0x7f40a800f450 con 0x7f40b01031c0 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 Processor -- start 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 -- start start 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adeaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adeb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adeb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e6ad51460 con 0x562e6aa222f0 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562e6ad0b590 con 0x562e6aa4b1b0 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40bc176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e6ad4d420 con 0x562e6aa2a4d0 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adeb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adeb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51344/0 (socket says 172.21.15.112:51344) 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40aeffd640 1 -- 172.21.15.112:0/1257412457 learned_addr learned my addr 172.21.15.112:0/1257412457 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:23.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40af7fe640 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adeaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.440+0000 7f40ae7fc640 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adeb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f4097fff640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40980afe00 0x7f40980b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40aeffd640 1 -- 172.21.15.112:0/1257412457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 msgr2=0x562e6adeb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40aeffd640 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adeb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40aeffd640 1 -- 172.21.15.112:0/1257412457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 msgr2=0x562e6adeaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40aeffd640 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adeaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40aeffd640 1 -- 172.21.15.112:0/1257412457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6ad180d0 con 0x562e6aa222f0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40af7fe640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40980afe00 0x7f40980b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f4097fff640 1 -- 172.21.15.112:0/52546963 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f40a80df420 con 0x7f40b01031c0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40aeffd640 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adeb030 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f40a80dc8b0 tx=0x7f40a80a2cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3a2c01ed5930f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40957fa640 1 -- 172.21.15.112:0/1257412457 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80a7130 con 0x562e6aa222f0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40957fa640 1 -- 172.21.15.112:0/1257412457 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a80a6e70 con 0x562e6aa222f0 2024-04-25T16:39:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.441+0000 7f40957fa640 1 -- 172.21.15.112:0/1257412457 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80ac8a0 con 0x562e6aa222f0 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 -- 172.21.15.112:0/1257412457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 msgr2=0x562e6adeb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40af7fe640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40980afe00 0x7f40980b2310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009430 tx=0x7f40a003d040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adeb030 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f40a80dc8b0 tx=0x7f40a80a2cd0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 -- 172.21.15.112:0/1257412457 shutdown_connections 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adeb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adeb030 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 --2- 172.21.15.112:0/1257412457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adeaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.442+0000 7f40bc176740 1 -- 172.21.15.112:0/1257412457 >> 172.21.15.112:0/1257412457 conn(0x562e6aa3f300 msgr2=0x562e6ade72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.443+0000 7f40bc176740 1 -- 172.21.15.112:0/1257412457 shutdown_connections 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.443+0000 7f40bc176740 1 -- 172.21.15.112:0/1257412457 wait complete. 2024-04-25T16:39:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.443+0000 7f40bc176740 1 -- start start 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.443+0000 7f40bc176740 1 Processor -- start 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adf51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562e6adf7c20 con 0x562e6aa4b1b0 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562e6acd8b60 con 0x562e6aa222f0 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562e6adf7da0 con 0x562e6aa2a4d0 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adf51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adf51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:33826/0 (socket says 172.21.15.112:33826) 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51360/0 (socket says 172.21.15.112:51360) 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 -- 172.21.15.112:0/1732878862 learned_addr learned my addr 172.21.15.112:0/1732878862 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40ae7fc640 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40bc176740 10 client.0 _ll_register_callbacks cb 0x562e6acfd2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 -- 172.21.15.112:0/1732878862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 msgr2=0x562e6adf5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 -- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 msgr2=0x562e6adf51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adf51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.444+0000 7f40af7fe640 1 -- 172.21.15.112:0/1732878862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6aa63210 con 0x562e6aa4b1b0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f40af7fe640 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfb110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f40a0035000 tx=0x7f40a0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6ed2f9b7ff4a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f4092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f4092ff5640 1 -- 172.21.15.112:0/1732878862 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0003d80 con 0x562e6aa4b1b0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f4092ff5640 1 -- 172.21.15.112:0/1732878862 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f40a0004050 con 0x562e6aa4b1b0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a80a6e70 con 0x562e6aa4b1b0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f4092ff5640 1 -- 172.21.15.112:0/1732878862 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f40a0004250 con 0x562e6aa4b1b0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f40bc176740 10 client.15465 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562e6ab2f620 con 0x562e6aa4b1b0 2024-04-25T16:39:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f4092ff5640 1 -- 172.21.15.112:0/1732878862 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a80a6e70 con 0x562e6aa4b1b0 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.445+0000 7f4092ff5640 1 -- 172.21.15.112:0/1732878862 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f40a0030ac0 con 0x562e6aa4b1b0 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40bc176740 10 client.15465 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40bc176740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40bc176740 20 client.15465 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40bc176740 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6aa09d80 0x562e6ae077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40bc176740 4 client.15465 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6ad5f070 con 0x562e6aa09d80 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.446+0000 7f40aeffd640 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6aa09d80 0x562e6ae077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.447+0000 7f40aeffd640 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6aa09d80 0x562e6ae077a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d352e0387eb52316 server_cookie=5f546af44123708f in_seq=0 out_seq=0 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.447+0000 7f4092ff5640 10 client.15465 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.447+0000 7f4092ff5640 1 -- 172.21.15.112:0/1732878862 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e6ad5f070 con 0x562e6aa09d80 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.447+0000 7f4092ff5640 10 client.15465 handle_command_reply: tid=0 2024-04-25T16:39:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.447+0000 7f40bc176740 1 client.15465 shutdown 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 msgr2=0x562e6adfb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfb110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f40a0035000 tx=0x7f40a0009800 comp rx=0 tx=0).stop 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 shutdown_connections 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6aa09d80 0x562e6ae077a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f4092ff5640 0 client.15465 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfb110 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 --2- 172.21.15.112:0/1732878862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adf51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 >> 172.21.15.112:0/1732878862 conn(0x562e6aa3f300 msgr2=0x562e6adea920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 shutdown_connections 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.448+0000 7f40bc176740 1 -- 172.21.15.112:0/1732878862 wait complete. 2024-04-25T16:39:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.449+0000 7f40bc176740 20 client.15465 trim_cache size 0 max 16384 2024-04-25T16:39:23.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 Processor -- start 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 -- start start 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adfb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adf3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b0170 con 0x562e6aa222f0 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ada40 con 0x562e6aa4b1b0 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40bc176740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ab890 con 0x562e6aa2a4d0 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adfb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adfb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51372/0 (socket says 172.21.15.112:51372) 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.473+0000 7f40aeffd640 1 -- 172.21.15.112:0/3295987931 learned_addr learned my addr 172.21.15.112:0/3295987931 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40af7fe640 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adf3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40ae7fc640 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40aeffd640 1 -- 172.21.15.112:0/3295987931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 msgr2=0x562e6adf3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40aeffd640 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40aeffd640 1 -- 172.21.15.112:0/3295987931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 msgr2=0x562e6adf3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40aeffd640 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adf3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40aeffd640 1 -- 172.21.15.112:0/3295987931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6adf0b70 con 0x562e6aa222f0 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f40aeffd640 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adfb010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40a800ffd0 tx=0x7f40a800fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156a012eef3fbc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f4094ff9640 1 -- 172.21.15.112:0/3295987931 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80a6ad0 con 0x562e6aa222f0 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f4094ff9640 1 -- 172.21.15.112:0/3295987931 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562e6ad5f070 con 0x562e6aa222f0 2024-04-25T16:39:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.474+0000 7f4094ff9640 1 -- 172.21.15.112:0/3295987931 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80a6cb0 con 0x562e6aa222f0 2024-04-25T16:39:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 -- 172.21.15.112:0/3295987931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 msgr2=0x562e6adfb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adfb010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f40a800ffd0 tx=0x7f40a800fdf0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 -- 172.21.15.112:0/3295987931 shutdown_connections 2024-04-25T16:39:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adf3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa4b1b0 0x562e6adf3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 --2- 172.21.15.112:0/3295987931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa222f0 0x562e6adfb010 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.475+0000 7f40bc176740 1 -- 172.21.15.112:0/3295987931 >> 172.21.15.112:0/3295987931 conn(0x562e6aa3f300 msgr2=0x562e6adee7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 -- 172.21.15.112:0/3295987931 shutdown_connections 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 -- 172.21.15.112:0/3295987931 wait complete. 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 -- start start 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 Processor -- start 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adfccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adfd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.476+0000 7f40bc176740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adfccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40aeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adfccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:33852/0 (socket says 172.21.15.112:33852) 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51382/0 (socket says 172.21.15.112:51382) 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40ae7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adfd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40aeffd640 1 -- 172.21.15.112:0/472695757 learned_addr learned my addr 172.21.15.112:0/472695757 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01abf00 con 0x562e6aa4b1b0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ad370 con 0x562e6aa222f0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ac5d0 con 0x562e6aa2a4d0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40bc176740 10 client.0 _ll_register_callbacks cb 0x7f40a0009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 -- 172.21.15.112:0/472695757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 msgr2=0x562e6adfd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adfd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 -- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 msgr2=0x562e6adfccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adfccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6ae6eb10 con 0x562e6aa4b1b0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40aeffd640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40ae7fc640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adfd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f40af7fe640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfd730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40a003a570 tx=0x7f40a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b83160fe82fe4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.477+0000 7f4092ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f4092ff5640 1 -- 172.21.15.112:0/472695757 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a002af40 con 0x562e6aa4b1b0 2024-04-25T16:39:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f4092ff5640 1 -- 172.21.15.112:0/472695757 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f40a0003d90 con 0x562e6aa4b1b0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e6ad5f070 con 0x562e6aa4b1b0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f4092ff5640 1 -- 172.21.15.112:0/472695757 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f40a002fdb0 con 0x562e6aa4b1b0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 10 client.15471 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x562e6ae6fc50 con 0x562e6aa4b1b0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f4092ff5640 1 -- 172.21.15.112:0/472695757 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a0050070 con 0x562e6aa4b1b0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f4092ff5640 1 -- 172.21.15.112:0/472695757 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f40a0055450 con 0x562e6aa4b1b0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 10 client.15471 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 20 client.15471 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.478+0000 7f40bc176740 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6a9df1c0 0x562e6ae72460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.479+0000 7f40bc176740 4 client.15471 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.479+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e6ad5f070 con 0x562e6a9df1c0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.479+0000 7f40aeffd640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6a9df1c0 0x562e6ae72460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.479+0000 7f4092ff5640 10 client.15471 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.479+0000 7f40aeffd640 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6a9df1c0 0x562e6ae72460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da0b5ee8d8ea69a server_cookie=e4db93a7040ae029 in_seq=0 out_seq=0 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.483+0000 7f4092ff5640 1 -- 172.21.15.112:0/472695757 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e6ad5f070 con 0x562e6a9df1c0 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.483+0000 7f4092ff5640 10 client.15471 handle_command_reply: tid=0 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.483+0000 7f40bc176740 1 client.15471 shutdown 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 msgr2=0x562e6adfd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfd730 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f40a003a570 tx=0x7f40a00046c0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 shutdown_connections 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562e6a9df1c0 0x562e6ae72460 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562e6aa2a4d0 0x562e6adfd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562e6aa4b1b0 0x562e6adfd730 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f4092ff5640 0 client.15471 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 --2- 172.21.15.112:0/472695757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562e6aa222f0 0x562e6adfccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 >> 172.21.15.112:0/472695757 conn(0x562e6aa3f300 msgr2=0x562e6adee7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 shutdown_connections 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 1 -- 172.21.15.112:0/472695757 wait complete. 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40bc176740 20 client.15471 trim_cache size 0 max 16384 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:39:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:39:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40980afe00 msgr2=0x7f40980b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40b52c6640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40980afe00 0x7f40980b2310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009430 tx=0x7f40a003d040 comp rx=0 tx=0).stop 2024-04-25T16:39:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 msgr2=0x7f40b0199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.484+0000 7f40b52c6640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b0199630 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40a800c8f0 tx=0x7f40a800cdb0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 shutdown_connections 2024-04-25T16:39:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40980afe00 0x7f40980b2310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b0108f00 0x7f40b019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40b0108a60 0x7f40b0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 --2- 172.21.15.112:0/52546963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40b01031c0 0x7f40b0199630 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 >> 172.21.15.112:0/52546963 conn(0x7f40b00781b0 msgr2=0x7f40b0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 shutdown_connections 2024-04-25T16:39:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:23.485+0000 7f40b52c6640 1 -- 172.21.15.112:0/52546963 wait complete. 2024-04-25T16:39:23.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:23 smithi112 ceph-mon[29122]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T16:39:23.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:23 smithi133 ceph-mon[32691]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T16:39:23.934 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:23 smithi116 ceph-mon[32712]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T16:39:25.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:25 smithi112 ceph-mon[29122]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:39:25.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:25 smithi133 ceph-mon[32691]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:39:25.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:25 smithi116 ceph-mon[32712]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T16:39:27.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:27 smithi112 ceph-mon[29122]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:27.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:27 smithi133 ceph-mon[32691]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:27.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:27 smithi116 ceph-mon[32712]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:29 smithi112 ceph-mon[29122]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:29.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:29 smithi133 ceph-mon[32691]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:29.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:29 smithi116 ceph-mon[32712]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:31 smithi112 ceph-mon[29122]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:31 smithi133 ceph-mon[32691]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:31.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:31 smithi116 ceph-mon[32712]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:32.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:32 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:39:32.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:32 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:39:32.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:32 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:39:33.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:33 smithi112 ceph-mon[29122]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:33.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:33 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:33.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:33 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:33 smithi133 ceph-mon[32691]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:33 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:33 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:33 smithi116 ceph-mon[32712]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:33 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:33.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:33 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:34.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:34 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:39:34.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:34 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:39:34.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:34 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:34.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:34 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:39:34.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:34 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:39:34.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:34 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:34.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:34 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:39:34.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:34 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:39:34.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:34 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:39:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:35 smithi112 ceph-mon[29122]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:35 smithi133 ceph-mon[32691]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:35 smithi116 ceph-mon[32712]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 -- 172.21.15.112:0/1834027025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441031c0 msgr2=0x7f764410f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 --2- 172.21.15.112:0/1834027025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441031c0 0x7f764410f4f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f763c009ca0 tx=0x7f763c02a8e0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 -- 172.21.15.112:0/1834027025 shutdown_connections 2024-04-25T16:39:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 --2- 172.21.15.112:0/1834027025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76441031c0 0x7f764410f4f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 --2- 172.21.15.112:0/1834027025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7644108f00 0x7f7644102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 --2- 172.21.15.112:0/1834027025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7644108a60 0x7f7644102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 -- 172.21.15.112:0/1834027025 >> 172.21.15.112:0/1834027025 conn(0x7f76440781b0 msgr2=0x7f76440785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 -- 172.21.15.112:0/1834027025 shutdown_connections 2024-04-25T16:39:36.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.633+0000 7f7649a92640 1 -- 172.21.15.112:0/1834027025 wait complete. 2024-04-25T16:39:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.634+0000 7f7649a92640 1 Processor -- start 2024-04-25T16:39:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 -- start start 2024-04-25T16:39:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76441031c0 0x7f7644199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 0x7f7644199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7644108f00 0x7f764419de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7644112cf0 con 0x7f7644108a60 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 0x7f7644199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 0x7f7644199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56174/0 (socket says 172.21.15.112:56174) 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7642ffd640 1 -- 172.21.15.112:0/3925309592 learned_addr learned my addr 172.21.15.112:0/3925309592 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f76437fe640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76441031c0 0x7f7644199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7644112b70 con 0x7f76441031c0 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7643fff640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7644108f00 0x7f764419de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.635+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7644112e70 con 0x7f7644108f00 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7642ffd640 1 -- 172.21.15.112:0/3925309592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7644108f00 msgr2=0x7f764419de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7644108f00 0x7f764419de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7642ffd640 1 -- 172.21.15.112:0/3925309592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76441031c0 msgr2=0x7f7644199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76441031c0 0x7f7644199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7642ffd640 1 -- 172.21.15.112:0/3925309592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764419e410 con 0x7f7644108a60 2024-04-25T16:39:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 0x7f7644199b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f763800c9a0 tx=0x7f763800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7640ff9640 1 -- 172.21.15.112:0/3925309592 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638007c70 con 0x7f7644108a60 2024-04-25T16:39:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7640ff9640 1 -- 172.21.15.112:0/3925309592 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7638007e10 con 0x7f7644108a60 2024-04-25T16:39:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.636+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764419e700 con 0x7f7644108a60 2024-04-25T16:39:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.637+0000 7f7640ff9640 1 -- 172.21.15.112:0/3925309592 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638018850 con 0x7f7644108a60 2024-04-25T16:39:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.637+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441a6100 con 0x7f7644108a60 2024-04-25T16:39:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.640+0000 7f7650a13740 1 Processor -- start 2024-04-25T16:39:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.640+0000 7f7650a13740 1 -- start start 2024-04-25T16:39:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7640ff9640 1 -- 172.21.15.112:0/3925309592 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 147805+0+0 (secure 0 0 0) 0x7f76380189f0 con 0x7f7644108a60 2024-04-25T16:39:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7650a13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b261b0 0x565505ec7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7650a13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ec5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7650a13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 0x565505ec5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7650a13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565505e2c460 con 0x565505b261b0 2024-04-25T16:39:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7650a13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565505de6590 con 0x565505b054d0 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7650a13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565505e28420 con 0x565505afd2f0 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 0x565505ec5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 0x565505ec5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:33758/0 (socket says 172.21.15.112:33758) 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 -- 172.21.15.112:0/3641288656 learned_addr learned my addr 172.21.15.112:0/3641288656 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7643fff640 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b261b0 0x565505ec7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f76437fe640 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ec5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 -- 172.21.15.112:0/3641288656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 msgr2=0x565505ec5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ec5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 -- 172.21.15.112:0/3641288656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b261b0 msgr2=0x565505ec7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b261b0 0x565505ec7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7642ffd640 1 -- 172.21.15.112:0/3641288656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565505df30d0 con 0x565505b054d0 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7643fff640 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b261b0 0x565505ec7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:39:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.641+0000 7f7640ff9640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f762c0afe80 0x7f762c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.642+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 0x565505ec5f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76380dc980 tx=0x7f76380e7620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1566623193db224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.642+0000 7f7643fff640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f762c0afe80 0x7f762c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.642+0000 7f7640ff9640 1 -- 172.21.15.112:0/3925309592 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f76380df350 con 0x7f7644108a60 2024-04-25T16:39:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.642+0000 7f7629ffb640 1 -- 172.21.15.112:0/3641288656 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380a3150 con 0x565505b054d0 2024-04-25T16:39:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.642+0000 7f7629ffb640 1 -- 172.21.15.112:0/3641288656 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76380a6da0 con 0x565505b054d0 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 -- 172.21.15.112:0/3641288656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 msgr2=0x565505ec5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7643fff640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f762c0afe80 0x7f762c0b2390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x565505ec2cd0 tx=0x7f763c031f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 0x565505ec5f00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76380dc980 tx=0x7f76380e7620 comp rx=0 tx=0).stop 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 -- 172.21.15.112:0/3641288656 shutdown_connections 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ec5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505b054d0 0x565505ec5f00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 --2- 172.21.15.112:0/3641288656 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b261b0 0x565505ec7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.643+0000 7f7650a13740 1 -- 172.21.15.112:0/3641288656 >> 172.21.15.112:0/3641288656 conn(0x565505b1a300 msgr2=0x565505ec22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.644+0000 7f7650a13740 1 -- 172.21.15.112:0/3641288656 shutdown_connections 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.644+0000 7f7650a13740 1 -- 172.21.15.112:0/3641288656 wait complete. 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.644+0000 7f7650a13740 1 -- start start 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.645+0000 7f7650a13740 1 Processor -- start 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.645+0000 7f7650a13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505ed4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.645+0000 7f7650a13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ed4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.645+0000 7f7650a13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b261b0 0x565505eda5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ed4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7642ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ed4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56190/0 (socket says 172.21.15.112:56190) 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505ed4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505ed4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:33762/0 (socket says 172.21.15.112:33762) 2024-04-25T16:39:36.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7642ffd640 1 -- 172.21.15.112:0/2465354006 learned_addr learned my addr 172.21.15.112:0/2465354006 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x565505ed70f0 con 0x565505b054d0 2024-04-25T16:39:36.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565505db3b60 con 0x565505afd2f0 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565505ed7270 con 0x565505b261b0 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7650a13740 10 client.0 _ll_register_callbacks cb 0x565505dd82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 -- 172.21.15.112:0/2465354006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b261b0 msgr2=0x565505eda5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7643fff640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b261b0 0x565505eda5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b261b0 0x565505eda5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 -- 172.21.15.112:0/2465354006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 msgr2=0x565505ed4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ed4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565505b3e210 con 0x565505afd2f0 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7642ffd640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ed4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7643fff640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b261b0 0x565505eda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f76437fe640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505ed4730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f764406c590 tx=0x7f7634012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cac97191b3618f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.646+0000 7f7622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7622ffd640 1 -- 172.21.15.112:0/2465354006 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634004030 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7622ffd640 1 -- 172.21.15.112:0/2465354006 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f76340041d0 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76380a6da0 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7622ffd640 1 -- 172.21.15.112:0/2465354006 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f76340043d0 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 10 client.25184 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x565505c0a620 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7622ffd640 1 -- 172.21.15.112:0/2465354006 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763400fda0 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7622ffd640 1 -- 172.21.15.112:0/2465354006 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7634003b90 con 0x565505afd2f0 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 10 client.25184 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 20 client.25184 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.647+0000 7f7650a13740 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505ae4d80 0x565505ee6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.648+0000 7f7650a13740 4 client.25184 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:39:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.648+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f76380a6da0 con 0x565505ae4d80 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.648+0000 7f7642ffd640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505ae4d80 0x565505ee6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.648+0000 7f7622ffd640 10 client.25184 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.648+0000 7f7642ffd640 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505ae4d80 0x565505ee6bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e599ff62ac36f79f server_cookie=3294a8f2316d6322 in_seq=0 out_seq=0 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7622ffd640 1 -- 172.21.15.112:0/2465354006 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f76380a6da0 con 0x565505ae4d80 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7622ffd640 10 client.25184 handle_command_reply: tid=0 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 client.25184 shutdown 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 msgr2=0x565505ed4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505ed4730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f764406c590 tx=0x7f7634012b60 comp rx=0 tx=0).stop 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 shutdown_connections 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505ae4d80 0x565505ee6bf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b261b0 0x565505eda5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7622ffd640 0 client.25184 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ed4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 --2- 172.21.15.112:0/2465354006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505ed4730 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.649+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 >> 172.21.15.112:0/2465354006 conn(0x565505b1a300 msgr2=0x565505ec0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.650+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 shutdown_connections 2024-04-25T16:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.650+0000 7f7650a13740 1 -- 172.21.15.112:0/2465354006 wait complete. 2024-04-25T16:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.650+0000 7f7650a13740 20 client.25184 trim_cache size 0 max 16384 2024-04-25T16:39:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.678+0000 7f7650a13740 1 Processor -- start 2024-04-25T16:39:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.678+0000 7f7650a13740 1 -- start start 2024-04-25T16:39:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7650a13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505ae4d80 0x565505ed6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7650a13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ed3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7650a13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ec22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7650a13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76441b0140 con 0x565505b054d0 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7650a13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76441ada10 con 0x565505ae4d80 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7650a13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76441ab860 con 0x565505afd2f0 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7643fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ec22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7643fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ec22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56206/0 (socket says 172.21.15.112:56206) 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7643fff640 1 -- 172.21.15.112:0/3734922083 learned_addr learned my addr 172.21.15.112:0/3734922083 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f76437fe640 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505ae4d80 0x565505ed6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f7642ffd640 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ed3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f76437fe640 1 -- 172.21.15.112:0/3734922083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 msgr2=0x565505ed3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f76437fe640 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ed3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f76437fe640 1 -- 172.21.15.112:0/3734922083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 msgr2=0x565505ec22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f76437fe640 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ec22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.679+0000 7f76437fe640 1 -- 172.21.15.112:0/3734922083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565505eca060 con 0x565505ae4d80 2024-04-25T16:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.680+0000 7f76437fe640 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505ae4d80 0x565505ed6c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f764406c590 tx=0x7f7634009d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48657a755b0095ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.680+0000 7f76297fa640 1 -- 172.21.15.112:0/3734922083 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7634028040 con 0x565505ae4d80 2024-04-25T16:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.680+0000 7f76297fa640 1 -- 172.21.15.112:0/3734922083 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763401c650 con 0x565505ae4d80 2024-04-25T16:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.680+0000 7f76297fa640 1 -- 172.21.15.112:0/3734922083 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400fb30 con 0x565505ae4d80 2024-04-25T16:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 -- 172.21.15.112:0/3734922083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505ae4d80 msgr2=0x565505ed6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505ae4d80 0x565505ed6c50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f764406c590 tx=0x7f7634009d90 comp rx=0 tx=0).stop 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 -- 172.21.15.112:0/3734922083 shutdown_connections 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505afd2f0 0x565505ed3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505b054d0 0x565505ec22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 --2- 172.21.15.112:0/3734922083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505ae4d80 0x565505ed6c50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 -- 172.21.15.112:0/3734922083 >> 172.21.15.112:0/3734922083 conn(0x565505b1a300 msgr2=0x565505ed3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 -- 172.21.15.112:0/3734922083 shutdown_connections 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.681+0000 7f7650a13740 1 -- 172.21.15.112:0/3734922083 wait complete. 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 -- start start 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 Processor -- start 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 0x565505edd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505edd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b054d0 0x565505edded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76441abed0 con 0x565505ae4d80 2024-04-25T16:39:36.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76441ad340 con 0x565505afd2f0 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.682+0000 7f7650a13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f76441ac5a0 con 0x565505b054d0 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 0x565505edd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7650a13740 10 client.0 _ll_register_callbacks cb 0x7f7634009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 0x565505edd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56218/0 (socket says 172.21.15.112:56218) 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 -- 172.21.15.112:0/1356659200 learned_addr learned my addr 172.21.15.112:0/1356659200 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7642ffd640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505edd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7643fff640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b054d0 0x565505edded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 -- 172.21.15.112:0/1356659200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b054d0 msgr2=0x565505edded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b054d0 0x565505edded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 -- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 msgr2=0x565505edd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505edd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 -- 172.21.15.112:0/1356659200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565505f4dfd0 con 0x565505ae4d80 2024-04-25T16:39:36.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7643fff640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b054d0 0x565505edded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f76437fe640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 0x565505edd450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7634009300 tx=0x7f763401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d966a8b8601956b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7622ffd640 1 -- 172.21.15.112:0/1356659200 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763401cd00 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763401c650 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.683+0000 7f7622ffd640 1 -- 172.21.15.112:0/1356659200 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f7634005050 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7622ffd640 1 -- 172.21.15.112:0/1356659200 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f7634005250 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 10 client.15489 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x565505f4f260 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7622ffd640 1 -- 172.21.15.112:0/1356659200 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f763401c650 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7622ffd640 1 -- 172.21.15.112:0/1356659200 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f7634003610 con 0x565505ae4d80 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 10 client.15489 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 10 client.15489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:36.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 20 client.15489 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505b261b0 0x565505f51930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 4 client.15489 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:39:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.684+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565505e3a070 con 0x565505b261b0 2024-04-25T16:39:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.685+0000 7f7642ffd640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505b261b0 0x565505f51930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.685+0000 7f7622ffd640 10 client.15489 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:36.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.685+0000 7f7642ffd640 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505b261b0 0x565505f51930 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3612e1b44af55671 server_cookie=b1b91f8e9aabce2e in_seq=0 out_seq=0 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7622ffd640 1 -- 172.21.15.112:0/1356659200 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565505e3a070 con 0x565505b261b0 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7622ffd640 10 client.15489 handle_command_reply: tid=0 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 client.15489 shutdown 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 msgr2=0x565505edd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 0x565505edd450 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7634009300 tx=0x7f763401ca30 comp rx=0 tx=0).stop 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 shutdown_connections 2024-04-25T16:39:36.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x565505b261b0 0x565505f51930 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565505b054d0 0x565505edded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7622ffd640 0 client.15489 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565505afd2f0 0x565505edd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 --2- 172.21.15.112:0/1356659200 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x565505ae4d80 0x565505edd450 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 >> 172.21.15.112:0/1356659200 conn(0x565505b1a300 msgr2=0x565505ed3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 shutdown_connections 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.689+0000 7f7650a13740 1 -- 172.21.15.112:0/1356659200 wait complete. 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7650a13740 20 client.15489 trim_cache size 0 max 16384 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:39:36.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:39:36.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f762c0afe80 msgr2=0x7f762c0b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f762c0afe80 0x7f762c0b2390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x565505ec2cd0 tx=0x7f763c031f90 comp rx=0 tx=0).stop 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 msgr2=0x7f7644199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 0x7f7644199b90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f763800c9a0 tx=0x7f763800ce60 comp rx=0 tx=0).stop 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 shutdown_connections 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f762c0afe80 0x7f762c0b2390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7644108f00 0x7f764419de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7644108a60 0x7f7644199b90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 --2- 172.21.15.112:0/3925309592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76441031c0 0x7f7644199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.690+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 >> 172.21.15.112:0/3925309592 conn(0x7f76440781b0 msgr2=0x7f7644100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.691+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 shutdown_connections 2024-04-25T16:39:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:36.691+0000 7f7649a92640 1 -- 172.21.15.112:0/3925309592 wait complete. 2024-04-25T16:39:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:37 smithi112 ceph-mon[29122]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:39:37.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:37 smithi133 ceph-mon[32691]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:39:37.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:37 smithi116 ceph-mon[32712]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:39:38.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:38 smithi133 ceph-mon[32691]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:39:38.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:38 smithi116 ceph-mon[32712]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:39:38.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:38 smithi112 ceph-mon[29122]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T16:39:40.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:40 smithi112 ceph-mon[29122]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:40.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:40 smithi133 ceph-mon[32691]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:41.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:40 smithi116 ceph-mon[32712]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:42.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:42 smithi133 ceph-mon[32691]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:43.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:42 smithi116 ceph-mon[32712]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:43.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:42 smithi112 ceph-mon[29122]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:44 smithi133 ceph-mon[32691]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:45.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:44 smithi116 ceph-mon[32712]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:45.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:44 smithi112 ceph-mon[29122]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:46 smithi133 ceph-mon[32691]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:46 smithi116 ceph-mon[32712]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:47.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:46 smithi112 ceph-mon[29122]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:48.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:48 smithi133 ceph-mon[32691]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:49.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:48 smithi116 ceph-mon[32712]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:49.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:48 smithi112 ceph-mon[29122]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.826+0000 7f471cde6640 1 -- 172.21.15.112:0/3114990746 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47080318a0 con 0x7f4718108f00 2024-04-25T16:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.826+0000 7f472004c640 1 -- 172.21.15.112:0/3114990746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4718108f00 msgr2=0x7f4718102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 --2- 172.21.15.112:0/3114990746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4718108f00 0x7f4718102b50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4708009900 tx=0x7f470802a410 comp rx=0 tx=0).stop 2024-04-25T16:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 -- 172.21.15.112:0/3114990746 shutdown_connections 2024-04-25T16:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 --2- 172.21.15.112:0/3114990746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47181031c0 0x7f471810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 --2- 172.21.15.112:0/3114990746 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4718108f00 0x7f4718102b50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 --2- 172.21.15.112:0/3114990746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4718108a60 0x7f4718102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 -- 172.21.15.112:0/3114990746 >> 172.21.15.112:0/3114990746 conn(0x7f47180781b0 msgr2=0x7f47180785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 -- 172.21.15.112:0/3114990746 shutdown_connections 2024-04-25T16:39:49.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 -- 172.21.15.112:0/3114990746 wait complete. 2024-04-25T16:39:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 Processor -- start 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.827+0000 7f472004c640 1 -- start start 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f472004c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47181031c0 0x7f4718198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f472004c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4718108a60 0x7f4718198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f472004c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 0x7f471819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f472004c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111b70 con 0x7f471819ce90 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f472004c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f47181119f0 con 0x7f4718108a60 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f472004c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4718111cf0 con 0x7f47181031c0 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 0x7f471819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 0x7f471819d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45200/0 (socket says 172.21.15.112:45200) 2024-04-25T16:39:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 -- 172.21.15.112:0/4292950701 learned_addr learned my addr 172.21.15.112:0/4292950701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471d5e7640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4718108a60 0x7f4718198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471dde8640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47181031c0 0x7f4718198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 -- 172.21.15.112:0/4292950701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47181031c0 msgr2=0x7f4718198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47181031c0 0x7f4718198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 -- 172.21.15.112:0/4292950701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4718108a60 msgr2=0x7f4718198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4718108a60 0x7f4718198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471e5e9640 1 -- 172.21.15.112:0/4292950701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471819d9a0 con 0x7f471819ce90 2024-04-25T16:39:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.828+0000 7f471d5e7640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4718108a60 0x7f4718198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:39:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.829+0000 7f471e5e9640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 0x7f471819d2a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4710009fd0 tx=0x7f471000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.829+0000 7f470effd640 1 -- 172.21.15.112:0/4292950701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710017070 con 0x7f471819ce90 2024-04-25T16:39:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.829+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471819dc90 con 0x7f471819ce90 2024-04-25T16:39:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.829+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a5490 con 0x7f471819ce90 2024-04-25T16:39:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.829+0000 7f470effd640 1 -- 172.21.15.112:0/4292950701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4710003e20 con 0x7f471819ce90 2024-04-25T16:39:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.829+0000 7f470effd640 1 -- 172.21.15.112:0/4292950701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710010460 con 0x7f471819ce90 2024-04-25T16:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.832+0000 7f4726fdc740 1 Processor -- start 2024-04-25T16:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.832+0000 7f470effd640 1 -- 172.21.15.112:0/4292950701 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 24) ==== 147805+0+0 (secure 0 0 0) 0x7f4710003970 con 0x7f471819ce90 2024-04-25T16:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 -- start start 2024-04-25T16:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a428af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a429030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a429570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55816a38f460 con 0x55816a0684d0 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55816a349590 con 0x55816a0602f0 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f4726fdc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55816a38b420 con 0x55816a0891b0 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.833+0000 7f470effd640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f47040afe80 0x7f47040b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a429570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f470effd640 1 -- 172.21.15.112:0/4292950701 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f47100df080 con 0x7f471819ce90 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471e5e9640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f47040afe80 0x7f47040b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a429570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45210/0 (socket says 172.21.15.112:45210) 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 -- 172.21.15.112:0/3403035431 learned_addr learned my addr 172.21.15.112:0/3403035431 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471dde8640 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a429030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471e5e9640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f47040afe80 0x7f47040b2390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47100dc5d0 tx=0x7f471000bea0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 -- 172.21.15.112:0/3403035431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 msgr2=0x55816a428af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a428af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 -- 172.21.15.112:0/3403035431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 msgr2=0x55816a429030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a429030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 -- 172.21.15.112:0/3403035431 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816a3560d0 con 0x55816a0684d0 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471dde8640 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a429030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.834+0000 7f471d5e7640 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a429570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4708009230 tx=0x7f4708000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241a9a2f0c9746eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.835+0000 7f4703fff640 1 -- 172.21.15.112:0/3403035431 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708035040 con 0x55816a0684d0 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.835+0000 7f4703fff640 1 -- 172.21.15.112:0/3403035431 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4708004030 con 0x55816a0684d0 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.835+0000 7f4703fff640 1 -- 172.21.15.112:0/3403035431 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47080046c0 con 0x55816a0684d0 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 -- 172.21.15.112:0/3403035431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 msgr2=0x55816a429570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a429570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4708009230 tx=0x7f4708000bc0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 -- 172.21.15.112:0/3403035431 shutdown_connections 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a429570 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a429030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 --2- 172.21.15.112:0/3403035431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a428af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 -- 172.21.15.112:0/3403035431 >> 172.21.15.112:0/3403035431 conn(0x55816a07d300 msgr2=0x55816a4252a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 -- 172.21.15.112:0/3403035431 shutdown_connections 2024-04-25T16:39:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.836+0000 7f4726fdc740 1 -- 172.21.15.112:0/3403035431 wait complete. 2024-04-25T16:39:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.837+0000 7f4726fdc740 1 -- start start 2024-04-25T16:39:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.837+0000 7f4726fdc740 1 Processor -- start 2024-04-25T16:39:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.837+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a4330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.837+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a433620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.837+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a43f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a43f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a43f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45218/0 (socket says 172.21.15.112:45218) 2024-04-25T16:39:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a433620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471d5e7640 1 -- 172.21.15.112:0/2608344166 learned_addr learned my addr 172.21.15.112:0/2608344166 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55816a435bc0 con 0x55816a0891b0 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55816a316b60 con 0x55816a0684d0 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55816a435d40 con 0x55816a0602f0 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 -- 172.21.15.112:0/2608344166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 msgr2=0x55816a4330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a4330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.845 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 -- 172.21.15.112:0/2608344166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 msgr2=0x55816a43f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a43f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55816a0a1210 con 0x55816a0684d0 2024-04-25T16:39:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a43f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:39:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f4726fdc740 10 client.0 _ll_register_callbacks cb 0x55816a33b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f471dde8640 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a433620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55816a426230 tx=0x7f4714009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d94eca3e2250576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f47017fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:39:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.838+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4708004030 con 0x55816a0684d0 2024-04-25T16:39:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f47017fa640 1 -- 172.21.15.112:0/2608344166 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714004030 con 0x55816a0684d0 2024-04-25T16:39:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f47017fa640 1 -- 172.21.15.112:0/2608344166 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f47140041d0 con 0x55816a0684d0 2024-04-25T16:39:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f47017fa640 1 -- 172.21.15.112:0/2608344166 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4714004430 con 0x55816a0684d0 2024-04-25T16:39:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 10 client.25199 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:49.847 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55816a16d620 con 0x55816a0684d0 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f47017fa640 1 -- 172.21.15.112:0/2608344166 <== mon.2 v2:172.21.15.116:3300/0 4 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f4714015990 con 0x55816a0684d0 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 10 client.25199 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 10 client.25199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 20 client.25199 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a047d80 0x55816a4455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 4 client.25199 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:39:49.848 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55816a39d070 con 0x55816a047d80 2024-04-25T16:39:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.839+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a047d80 0x55816a4455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.840+0000 7f47017fa640 10 client.25199 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.840+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a047d80 0x55816a4455b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11d162934c2b96d server_cookie=8f258f82194d5ff4 in_seq=0 out_seq=0 2024-04-25T16:39:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f47017fa640 1 -- 172.21.15.112:0/2608344166 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55816a39d070 con 0x55816a047d80 2024-04-25T16:39:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f47017fa640 10 client.25199 handle_command_reply: tid=0 2024-04-25T16:39:49.849 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 client.25199 shutdown 2024-04-25T16:39:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 msgr2=0x55816a433620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a433620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55816a426230 tx=0x7f4714009e50 comp rx=0 tx=0).stop 2024-04-25T16:39:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 shutdown_connections 2024-04-25T16:39:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a43f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a433620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.850 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a047d80 0x55816a4455b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f47017fa640 0 client.25199 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2608344166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a4330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 >> 172.21.15.112:0/2608344166 conn(0x55816a07d300 msgr2=0x55816a42a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 shutdown_connections 2024-04-25T16:39:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 1 -- 172.21.15.112:0/2608344166 wait complete. 2024-04-25T16:39:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.841+0000 7f4726fdc740 20 client.25199 trim_cache size 0 max 16384 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 Processor -- start 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 -- start start 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a42d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a43ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a435020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc011070 con 0x55816a0684d0 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc00e940 con 0x55816a0602f0 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f4726fdc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc00c790 con 0x55816a0891b0 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a42d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f471dde8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a43ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f471dde8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a43ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45234/0 (socket says 172.21.15.112:45234) 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f471dde8640 1 -- 172.21.15.112:0/2984813320 learned_addr learned my addr 172.21.15.112:0/2984813320 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.871+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a435020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471d5e7640 1 -- 172.21.15.112:0/2984813320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 msgr2=0x55816a435020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a435020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471d5e7640 1 -- 172.21.15.112:0/2984813320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 msgr2=0x55816a43ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a43ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471d5e7640 1 -- 172.21.15.112:0/2984813320 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816a4263f0 con 0x55816a0602f0 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471dde8640 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a43ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a42d750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55816a445d50 tx=0x7f4708009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e73b502a31453c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f47037fe640 1 -- 172.21.15.112:0/2984813320 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47080041e0 con 0x55816a0602f0 2024-04-25T16:39:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f47037fe640 1 -- 172.21.15.112:0/2984813320 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4708004380 con 0x55816a0602f0 2024-04-25T16:39:49.873 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.872+0000 7f47037fe640 1 -- 172.21.15.112:0/2984813320 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708003c10 con 0x55816a0602f0 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 -- 172.21.15.112:0/2984813320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 msgr2=0x55816a42d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a42d750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55816a445d50 tx=0x7f4708009260 comp rx=0 tx=0).stop 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 -- 172.21.15.112:0/2984813320 shutdown_connections 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0891b0 0x55816a435020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0684d0 0x55816a43ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2984813320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0602f0 0x55816a42d750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 -- 172.21.15.112:0/2984813320 >> 172.21.15.112:0/2984813320 conn(0x55816a07d300 msgr2=0x55816a4242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 -- 172.21.15.112:0/2984813320 shutdown_connections 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.873+0000 7f4726fdc740 1 -- 172.21.15.112:0/2984813320 wait complete. 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.874+0000 7f4726fdc740 1 -- start start 2024-04-25T16:39:49.877 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.874+0000 7f4726fdc740 1 Processor -- start 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.874+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a43b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a4366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f4726fdc740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a436be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f4726fdc740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc00ce00 con 0x55816a0891b0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f4726fdc740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc00e270 con 0x55816a0684d0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f4726fdc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc00d4d0 con 0x55816a0602f0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f4726fdc740 10 client.0 _ll_register_callbacks cb 0x7f471400b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a43b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a43b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:46194/0 (socket says 172.21.15.112:46194) 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 -- 172.21.15.112:0/2279316723 learned_addr learned my addr 172.21.15.112:0/2279316723 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a436be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471dde8640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a4366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.878 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 -- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 msgr2=0x55816a4366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a4366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 -- 172.21.15.112:0/2279316723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 msgr2=0x55816a436be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a436be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 -- 172.21.15.112:0/2279316723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55816a437120 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f471d5e7640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a43b2d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55816a42a3c0 tx=0x7f4708005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=734ae15da4f9bc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.875+0000 7f47017fa640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f47017fa640 1 -- 172.21.15.112:0/2279316723 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470806b060 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4708004380 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f47017fa640 1 -- 172.21.15.112:0/2279316723 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f47080665e0 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f47017fa640 1 -- 172.21.15.112:0/2279316723 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4708035070 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 10 client.25159 fetch_fsmap learned FSMap version 22 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55816a4ad530 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f47017fa640 1 -- 172.21.15.112:0/2279316723 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4708004380 con 0x55816a0602f0 2024-04-25T16:39:49.879 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f47017fa640 1 -- 172.21.15.112:0/2279316723 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f470803a070 con 0x55816a0602f0 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 10 client.25159 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 10 client.25159 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 20 client.25159 populate_metadata read hostname 'smithi112' 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a01d1c0 0x55816a4af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 4 client.25159 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.876+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55816a396ee0 con 0x55816a01d1c0 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.877+0000 7f471dde8640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a01d1c0 0x55816a4af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.877+0000 7f47017fa640 10 client.25159 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.877+0000 7f471dde8640 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a01d1c0 0x55816a4af7c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f301e2b43591b114 server_cookie=e7dc6ec7d26bdc22 in_seq=0 out_seq=0 2024-04-25T16:39:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f47017fa640 1 -- 172.21.15.112:0/2279316723 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55816a396ee0 con 0x55816a01d1c0 2024-04-25T16:39:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f47017fa640 10 client.25159 handle_command_reply: tid=0 2024-04-25T16:39:49.882 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 client.25159 shutdown 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 msgr2=0x55816a43b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a43b2d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55816a42a3c0 tx=0x7f4708005760 comp rx=0 tx=0).stop 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 shutdown_connections 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55816a0891b0 0x55816a436be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55816a0684d0 0x55816a4366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55816a01d1c0 0x55816a4af7c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 --2- 172.21.15.112:0/2279316723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55816a0602f0 0x55816a43b2d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 >> 172.21.15.112:0/2279316723 conn(0x55816a07d300 msgr2=0x55816a4242f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f47017fa640 0 client.25159 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 shutdown_connections 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 1 -- 172.21.15.112:0/2279316723 wait complete. 2024-04-25T16:39:49.883 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.882+0000 7f4726fdc740 20 client.25159 trim_cache size 0 max 16384 2024-04-25T16:39:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:39:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:39:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:39:49.884 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f47040afe80 msgr2=0x7f47040b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f47040afe80 0x7f47040b2390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47100dc5d0 tx=0x7f471000bea0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 msgr2=0x7f471819d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 0x7f471819d2a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4710009fd0 tx=0x7f471000b260 comp rx=0 tx=0).stop 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 shutdown_connections 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f47040afe80 0x7f47040b2390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.885 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f471819ce90 0x7f471819d2a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4718108a60 0x7f4718198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 --2- 172.21.15.112:0/4292950701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47181031c0 0x7f4718198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:39:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 >> 172.21.15.112:0/4292950701 conn(0x7f47180781b0 msgr2=0x7f4718104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:39:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 shutdown_connections 2024-04-25T16:39:49.886 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:39:49.883+0000 7f472004c640 1 -- 172.21.15.112:0/4292950701 wait complete. 2024-04-25T16:39:50.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:50 smithi133 ceph-mon[32691]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:51.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:50 smithi116 ceph-mon[32712]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:51.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:50 smithi112 ceph-mon[29122]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:52.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:52 smithi133 ceph-mon[32691]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T16:39:53.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:52 smithi116 ceph-mon[32712]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T16:39:53.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:52 smithi112 ceph-mon[29122]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T16:39:53.731 INFO:tasks.workunit.client.0.smithi112.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: git switch -c 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr:Or undo this operation with: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: git switch - 2024-04-25T16:39:53.732 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.733 INFO:tasks.workunit.client.0.smithi112.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T16:39:53.733 INFO:tasks.workunit.client.0.smithi112.stderr: 2024-04-25T16:39:53.733 INFO:tasks.workunit.client.0.smithi112.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T16:39:53.738 DEBUG:teuthology.orchestra.run.smithi112:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T16:39:53.801 INFO:tasks.workunit.client.0.smithi112.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T16:39:53.803 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T16:39:53.803 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T16:39:54.429 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T16:39:54.632 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T16:39:54.632 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T16:39:54.633 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T16:39:54.633 INFO:tasks.workunit.client.0.smithi112.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T16:39:54.633 INFO:tasks.workunit.client.0.smithi112.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T16:39:54.633 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:39:54.633 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T16:39:54.694 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-25T16:39:54.695 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-25T16:39:54.695 DEBUG:teuthology.orchestra.run.smithi112:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-25T16:39:54.756 INFO:tasks.workunit.client.0.smithi112.stderr:+ dbench 1 2024-04-25T16:39:54.765 INFO:tasks.workunit.client.0.smithi112.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T16:39:54.765 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T16:39:54.765 INFO:tasks.workunit.client.0.smithi112.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T16:39:54.765 INFO:tasks.workunit.client.0.smithi112.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-25T16:39:54.866 INFO:tasks.workunit.client.0.smithi112.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-25T16:39:54.866 INFO:tasks.workunit.client.0.smithi112.stdout:releasing clients 2024-04-25T16:39:54.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:54 smithi133 ceph-mon[32691]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:55.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:54 smithi116 ceph-mon[32712]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:55.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:54 smithi112 ceph-mon[29122]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T16:39:55.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 3024 33.54 MB/sec warmup 1 sec latency 10.699 ms 2024-04-25T16:39:56.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 5918 23.51 MB/sec warmup 2 sec latency 12.917 ms 2024-04-25T16:39:56.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:56 smithi133 ceph-mon[32691]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:57.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:56 smithi116 ceph-mon[32712]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:57.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:56 smithi112 ceph-mon[29122]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:57.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 8170 20.38 MB/sec warmup 3 sec latency 15.805 ms 2024-04-25T16:39:58.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 11225 19.58 MB/sec warmup 4 sec latency 10.247 ms 2024-04-25T16:39:58.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:39:58 smithi133 ceph-mon[32691]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:59.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:39:58 smithi116 ceph-mon[32712]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:59.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:39:58 smithi112 ceph-mon[29122]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T16:39:59.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 14304 19.22 MB/sec warmup 5 sec latency 18.737 ms 2024-04-25T16:40:00.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 15725 17.10 MB/sec warmup 6 sec latency 21.109 ms 2024-04-25T16:40:00.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:00 smithi133 ceph-mon[32691]: pgmap v49: 97 pgs: 97 active+clean; 6.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 516 KiB/s wr, 67 op/s 2024-04-25T16:40:00.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:00 smithi133 ceph-mon[32691]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T16:40:00.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:00 smithi133 ceph-mon[32691]: overall HEALTH_OK 2024-04-25T16:40:01.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:00 smithi116 ceph-mon[32712]: pgmap v49: 97 pgs: 97 active+clean; 6.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 516 KiB/s wr, 67 op/s 2024-04-25T16:40:01.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:00 smithi116 ceph-mon[32712]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T16:40:01.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:00 smithi116 ceph-mon[32712]: overall HEALTH_OK 2024-04-25T16:40:01.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:00 smithi112 ceph-mon[29122]: pgmap v49: 97 pgs: 97 active+clean; 6.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 516 KiB/s wr, 67 op/s 2024-04-25T16:40:01.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:00 smithi112 ceph-mon[29122]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T16:40:01.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:00 smithi112 ceph-mon[29122]: overall HEALTH_OK 2024-04-25T16:40:01.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 18524 16.71 MB/sec warmup 7 sec latency 12.816 ms 2024-04-25T16:40:02.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 21569 16.93 MB/sec warmup 8 sec latency 69.218 ms 2024-04-25T16:40:02.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:02 smithi133 ceph-mon[32691]: pgmap v50: 97 pgs: 97 active+clean; 6.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 516 KiB/s wr, 65 op/s 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 -- 172.21.15.112:0/1521163083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c1020b0 msgr2=0x7f8b4c1024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/1521163083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c1020b0 0x7f8b4c1024c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802a410 comp rx=0 tx=0).stop 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 -- 172.21.15.112:0/1521163083 shutdown_connections 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/1521163083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c10e5b0 0x7f8b4c1109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/1521163083 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c1020b0 0x7f8b4c1024c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/1521163083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b4c109c00 0x7f8b4c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 -- 172.21.15.112:0/1521163083 >> 172.21.15.112:0/1521163083 conn(0x7f8b4c0fd8c0 msgr2=0x7f8b4c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 -- 172.21.15.112:0/1521163083 shutdown_connections 2024-04-25T16:40:02.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.988+0000 7f8b52bc5640 1 -- 172.21.15.112:0/1521163083 wait complete. 2024-04-25T16:40:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 Processor -- start 2024-04-25T16:40:02.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 -- start start 2024-04-25T16:40:02.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c1020b0 0x7f8b4c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 0x7f8b4c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b4c10e5b0 0x7f8b4c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c113d70 con 0x7f8b4c109c00 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c113bf0 con 0x7f8b4c10e5b0 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b52bc5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c113ef0 con 0x7f8b4c1020b0 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 0x7f8b4c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 0x7f8b4c19acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56130/0 (socket says 172.21.15.112:56130) 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b51162640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b4c10e5b0 0x7f8b4c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 -- 172.21.15.112:0/4096911632 learned_addr learned my addr 172.21.15.112:0/4096911632 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b50961640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c1020b0 0x7f8b4c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:02.990 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 -- 172.21.15.112:0/4096911632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c1020b0 msgr2=0x7f8b4c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c1020b0 0x7f8b4c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 -- 172.21.15.112:0/4096911632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b4c10e5b0 msgr2=0x7f8b4c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b4c10e5b0 0x7f8b4c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 -- 172.21.15.112:0/4096911632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c19f5d0 con 0x7f8b4c109c00 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.989+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 0x7f8b4c19acd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b38032fd0 tx=0x7f8b38031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.990+0000 7f8b49ffb640 1 -- 172.21.15.112:0/4096911632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3803f030 con 0x7f8b4c109c00 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.990+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c19f860 con 0x7f8b4c109c00 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.990+0000 7f8b49ffb640 1 -- 172.21.15.112:0/4096911632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b38004490 con 0x7f8b4c109c00 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.990+0000 7f8b49ffb640 1 -- 172.21.15.112:0/4096911632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38035070 con 0x7f8b4c109c00 2024-04-25T16:40:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.990+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c19fd50 con 0x7f8b4c109c00 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.992+0000 7f8b49ffb640 1 -- 172.21.15.112:0/4096911632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f8b38042030 con 0x7f8b4c109c00 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 Processor -- start 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 -- start start 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cef2f0 0x5594080b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55940801e460 con 0x559407cf74d0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559407fd8590 con 0x559407cef2f0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b59a78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55940801a420 con 0x559407d181b0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b50961640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cef2f0 0x5594080b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b51162640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56138/0 (socket says 172.21.15.112:56138) 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 -- 172.21.15.112:0/3130758464 learned_addr learned my addr 172.21.15.112:0/3130758464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b49ffb640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8b3c0afc50 0x7f8b3c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 -- 172.21.15.112:0/3130758464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 msgr2=0x5594080b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 -- 172.21.15.112:0/3130758464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cef2f0 msgr2=0x5594080b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cef2f0 0x5594080b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b4bfff640 1 -- 172.21.15.112:0/3130758464 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559407fe50d0 con 0x559407cf74d0 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b51162640 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b50961640 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cef2f0 0x5594080b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.993+0000 7f8b51162640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8b3c0afc50 0x7f8b3c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.994+0000 7f8b49ffb640 1 -- 172.21.15.112:0/4096911632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f8b380ff360 con 0x7f8b4c109c00 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.994+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080b8400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b380fca00 tx=0x7f8b3808da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fee3a3142da323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.994+0000 7f8b51162640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8b3c0afc50 0x7f8b3c0b2160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x5594080b4c90 tx=0x7f8b44004a40 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.994+0000 7f8b36ffd640 1 -- 172.21.15.112:0/3130758464 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b380c6ee0 con 0x559407cf74d0 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.994+0000 7f8b36ffd640 1 -- 172.21.15.112:0/3130758464 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b380c70a0 con 0x559407cf74d0 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.994+0000 7f8b36ffd640 1 -- 172.21.15.112:0/3130758464 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38092d00 con 0x559407cf74d0 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 -- 172.21.15.112:0/3130758464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 msgr2=0x5594080b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080b8400 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8b380fca00 tx=0x7f8b3808da60 comp rx=0 tx=0).stop 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 -- 172.21.15.112:0/3130758464 shutdown_connections 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080b8400 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cef2f0 0x5594080b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3130758464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 -- 172.21.15.112:0/3130758464 >> 172.21.15.112:0/3130758464 conn(0x559407d0c300 msgr2=0x5594080b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 -- 172.21.15.112:0/3130758464 shutdown_connections 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 -- 172.21.15.112:0/3130758464 wait complete. 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 -- start start 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.995+0000 7f8b59a78740 1 Processor -- start 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cf74d0 0x5594080c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x5594080cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5594080c8e50 con 0x559407cef2f0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559407fa5b60 con 0x559407d181b0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5594080c8fd0 con 0x559407cf74d0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b59a78740 10 client.0 _ll_register_callbacks cb 0x559407fca2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56152/0 (socket says 172.21.15.112:56152) 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 -- 172.21.15.112:0/3519148830 learned_addr learned my addr 172.21.15.112:0/3519148830 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b51162640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x5594080cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cf74d0 0x5594080c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 -- 172.21.15.112:0/3519148830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cf74d0 msgr2=0x5594080c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cf74d0 0x5594080c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 -- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 msgr2=0x5594080cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x5594080cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 -- 172.21.15.112:0/3519148830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559407d30210 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b50961640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080c65d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8b40012700 tx=0x7f8b40012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad7989d556d82df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b2bfff640 1 -- 172.21.15.112:0/3519148830 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40004050 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.996+0000 7f8b2bfff640 1 -- 172.21.15.112:0/3519148830 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f8b400041f0 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b380c70a0 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b2bfff640 1 -- 172.21.15.112:0/3519148830 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8b400043f0 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 10 client.15519 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559407dfc620 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b2bfff640 1 -- 172.21.15.112:0/3519148830 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b380c70a0 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b2bfff640 1 -- 172.21.15.112:0/3519148830 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8b40003b20 con 0x559407cef2f0 2024-04-25T16:40:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 10 client.15519 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 20 client.15519 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cd6d80 0x5594080d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 4 client.15519 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55940802c070 con 0x559407cd6d80 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.997+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cd6d80 0x5594080d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.998+0000 7f8b2bfff640 10 client.15519 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.998+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cd6d80 0x5594080d8840 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a023078c3705ab8 server_cookie=2808b11e85267aae in_seq=0 out_seq=0 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b2bfff640 1 -- 172.21.15.112:0/3519148830 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55940802c070 con 0x559407cd6d80 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b2bfff640 10 client.15519 handle_command_reply: tid=0 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 client.15519 shutdown 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 msgr2=0x5594080c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080c65d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8b40012700 tx=0x7f8b40012bc0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 shutdown_connections 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cd6d80 0x5594080d8840 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b2bfff640 0 client.15519 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cf74d0 0x5594080c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x5594080cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 --2- 172.21.15.112:0/3519148830 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080c65d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 >> 172.21.15.112:0/3519148830 conn(0x559407d0c300 msgr2=0x5594080b9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 shutdown_connections 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 1 -- 172.21.15.112:0/3519148830 wait complete. 2024-04-25T16:40:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:02.999+0000 7f8b59a78740 20 client.15519 trim_cache size 0 max 16384 2024-04-25T16:40:03.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:02 smithi116 ceph-mon[32712]: pgmap v50: 97 pgs: 97 active+clean; 6.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 516 KiB/s wr, 65 op/s 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 Processor -- start 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 -- start start 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cf74d0 0x5594080b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1b0270 con 0x559407cef2f0 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1adb40 con 0x559407cf74d0 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.014+0000 7f8b59a78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1ab990 con 0x559407d181b0 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56162/0 (socket says 172.21.15.112:56162) 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 -- 172.21.15.112:0/451711119 learned_addr learned my addr 172.21.15.112:0/451711119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cf74d0 0x5594080b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b51162640 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 -- 172.21.15.112:0/451711119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 msgr2=0x5594080b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 -- 172.21.15.112:0/451711119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cf74d0 msgr2=0x5594080b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cf74d0 0x5594080b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 -- 172.21.15.112:0/451711119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594080c76f0 con 0x559407cef2f0 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b50961640 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080cc140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b40000c00 tx=0x7f8b400107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371af004369c6aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b367fc640 1 -- 172.21.15.112:0/451711119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b40029040 con 0x559407cef2f0 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b367fc640 1 -- 172.21.15.112:0/451711119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b40010e20 con 0x559407cef2f0 2024-04-25T16:40:03.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.015+0000 7f8b367fc640 1 -- 172.21.15.112:0/451711119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000fdc0 con 0x559407cef2f0 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 -- 172.21.15.112:0/451711119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 msgr2=0x5594080cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080cc140 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8b40000c00 tx=0x7f8b400107b0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 -- 172.21.15.112:0/451711119 shutdown_connections 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407d181b0 0x5594080b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407cf74d0 0x5594080b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 --2- 172.21.15.112:0/451711119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cef2f0 0x5594080cc140 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 -- 172.21.15.112:0/451711119 >> 172.21.15.112:0/451711119 conn(0x559407d0c300 msgr2=0x5594080b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 -- 172.21.15.112:0/451711119 shutdown_connections 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 -- 172.21.15.112:0/451711119 wait complete. 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 -- start start 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.016+0000 7f8b59a78740 1 Processor -- start 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cef2f0 0x5594080cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080cde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x559408138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1ac000 con 0x559407cf74d0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1ad470 con 0x559407d181b0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c1ac6d0 con 0x559407cef2f0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b59a78740 10 client.0 _ll_register_callbacks cb 0x7f8b40009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56170/0 (socket says 172.21.15.112:56170) 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 -- 172.21.15.112:0/2361938575 learned_addr learned my addr 172.21.15.112:0/2361938575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b51162640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x559408138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b50961640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cef2f0 0x5594080cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 -- 172.21.15.112:0/2361938575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cef2f0 msgr2=0x5594080cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cef2f0 0x5594080cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 -- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 msgr2=0x559408138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x559408138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.017+0000 7f8b4bfff640 1 -- 172.21.15.112:0/2361938575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594080c7280 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b4bfff640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080cde00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8b380fd090 tx=0x7f8b38009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1656c6885cad9586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b2bfff640 1 -- 172.21.15.112:0/2361938575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38049040 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b2bfff640 1 -- 172.21.15.112:0/2361938575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f8b38003d30 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b40010e20 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b2bfff640 1 -- 172.21.15.112:0/2361938575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f8b38094e60 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 10 client.15525 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x559408140360 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b2bfff640 1 -- 172.21.15.112:0/2361938575 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b40010e20 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b2bfff640 1 -- 172.21.15.112:0/2361938575 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f8b380fcd10 con 0x559407cf74d0 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 10 client.15525 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 20 client.15525 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cac1c0 0x559408142820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 4 client.15525 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:40:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559408025ee0 con 0x559407cac1c0 2024-04-25T16:40:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.018+0000 7f8b50961640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cac1c0 0x559408142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.019+0000 7f8b2bfff640 10 client.15525 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.019+0000 7f8b50961640 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cac1c0 0x559408142820 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127e8ab719901285 server_cookie=e563214988b3f92d in_seq=0 out_seq=0 2024-04-25T16:40:03.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:02 smithi112 ceph-mon[29122]: pgmap v50: 97 pgs: 97 active+clean; 6.7 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 516 KiB/s wr, 65 op/s 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b2bfff640 1 -- 172.21.15.112:0/2361938575 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559408025ee0 con 0x559407cac1c0 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b2bfff640 10 client.15525 handle_command_reply: tid=0 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 client.15525 shutdown 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 msgr2=0x5594080cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080cde00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8b380fd090 tx=0x7f8b38009260 comp rx=0 tx=0).stop 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 shutdown_connections 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559407d181b0 0x559408138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559407cf74d0 0x5594080cde00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559407cac1c0 0x559408142820 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 --2- 172.21.15.112:0/2361938575 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559407cef2f0 0x5594080cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b2bfff640 0 client.15525 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 >> 172.21.15.112:0/2361938575 conn(0x559407d0c300 msgr2=0x5594080b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 shutdown_connections 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 1 -- 172.21.15.112:0/2361938575 wait complete. 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.121+0000 7f8b59a78740 20 client.15525 trim_cache size 0 max 16384 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:40:03.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:40:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8b3c0afc50 msgr2=0x7f8b3c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8b3c0afc50 0x7f8b3c0b2160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x5594080b4c90 tx=0x7f8b44004a40 comp rx=0 tx=0).stop 2024-04-25T16:40:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 msgr2=0x7f8b4c19acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 0x7f8b4c19acd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8b38032fd0 tx=0x7f8b38031a20 comp rx=0 tx=0).stop 2024-04-25T16:40:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 shutdown_connections 2024-04-25T16:40:03.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8b3c0afc50 0x7f8b3c0b2160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b4c10e5b0 0x7f8b4c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8b4c109c00 0x7f8b4c19acd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 --2- 172.21.15.112:0/4096911632 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c1020b0 0x7f8b4c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 >> 172.21.15.112:0/4096911632 conn(0x7f8b4c0fd8c0 msgr2=0x7f8b4c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 shutdown_connections 2024-04-25T16:40:03.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:03.122+0000 7f8b52bc5640 1 -- 172.21.15.112:0/4096911632 wait complete. 2024-04-25T16:40:03.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 24076 16.30 MB/sec warmup 9 sec latency 54.696 ms 2024-04-25T16:40:04.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 26524 16.01 MB/sec warmup 10 sec latency 12.320 ms 2024-04-25T16:40:04.896 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:04 smithi112 ceph-mon[29122]: pgmap v51: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-04-25T16:40:04.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:04 smithi133 ceph-mon[32691]: pgmap v51: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-04-25T16:40:05.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:04 smithi116 ceph-mon[32712]: pgmap v51: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.3 MiB/s wr, 245 op/s 2024-04-25T16:40:05.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 29197 15.85 MB/sec warmup 11 sec latency 21.138 ms 2024-04-25T16:40:06.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 32348 16.09 MB/sec warmup 12 sec latency 6.782 ms 2024-04-25T16:40:06.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:06 smithi133 ceph-mon[32691]: pgmap v52: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-25T16:40:07.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:06 smithi116 ceph-mon[32712]: pgmap v52: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-25T16:40:07.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:06 smithi112 ceph-mon[29122]: pgmap v52: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-25T16:40:07.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 35333 15.97 MB/sec warmup 13 sec latency 22.140 ms 2024-04-25T16:40:08.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 37909 15.66 MB/sec warmup 14 sec latency 35.555 ms 2024-04-25T16:40:08.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:08 smithi133 ceph-mon[32691]: pgmap v53: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-25T16:40:09.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:08 smithi116 ceph-mon[32712]: pgmap v53: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-25T16:40:09.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:08 smithi112 ceph-mon[29122]: pgmap v53: 97 pgs: 97 active+clean; 10 MiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 1.3 MiB/s wr, 243 op/s 2024-04-25T16:40:09.866 INFO:tasks.workunit.client.0.smithi112.stdout: 1 40239 15.56 MB/sec warmup 15 sec latency 19.714 ms 2024-04-25T16:40:10.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 43147 15.65 MB/sec warmup 16 sec latency 29.931 ms 2024-04-25T16:40:10.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:10 smithi133 ceph-mon[32691]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.0 MiB/s wr, 330 op/s 2024-04-25T16:40:11.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:10 smithi116 ceph-mon[32712]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.0 MiB/s wr, 330 op/s 2024-04-25T16:40:11.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:10 smithi112 ceph-mon[29122]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.0 MiB/s wr, 330 op/s 2024-04-25T16:40:11.869 INFO:tasks.workunit.client.0.smithi112.stdout: 1 46290 15.78 MB/sec warmup 17 sec latency 19.256 ms 2024-04-25T16:40:12.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 48912 15.59 MB/sec warmup 18 sec latency 20.484 ms 2024-04-25T16:40:12.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:12 smithi133 ceph-mon[32691]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.5 MiB/s wr, 268 op/s 2024-04-25T16:40:13.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:12 smithi116 ceph-mon[32712]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.5 MiB/s wr, 268 op/s 2024-04-25T16:40:13.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:12 smithi112 ceph-mon[29122]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 1.5 MiB/s wr, 268 op/s 2024-04-25T16:40:13.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 51196 15.46 MB/sec warmup 19 sec latency 21.683 ms 2024-04-25T16:40:14.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 53958 15.40 MB/sec warmup 20 sec latency 12.469 ms 2024-04-25T16:40:14.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:14 smithi133 ceph-mon[32691]: pgmap v56: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-04-25T16:40:15.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:14 smithi116 ceph-mon[32712]: pgmap v56: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-04-25T16:40:15.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:14 smithi112 ceph-mon[29122]: pgmap v56: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 354 op/s 2024-04-25T16:40:15.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 57139 15.57 MB/sec warmup 21 sec latency 9.393 ms 2024-04-25T16:40:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/3552841786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 msgr2=0x7fd3cc102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/3552841786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc102b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009900 tx=0x7fd3bc02a410 comp rx=0 tx=0).stop 2024-04-25T16:40:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/3552841786 shutdown_connections 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/3552841786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3cc1031c0 0x7fd3cc10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/3552841786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc102b50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/3552841786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3cc108a60 0x7fd3cc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/3552841786 >> 172.21.15.112:0/3552841786 conn(0x7fd3cc0781b0 msgr2=0x7fd3cc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/3552841786 shutdown_connections 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.226+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/3552841786 wait complete. 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 Processor -- start 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 -- start start 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3cc1031c0 0x7fd3cc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3cc108a60 0x7fd3cc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc112c80 con 0x7fd3cc108f00 2024-04-25T16:40:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc112b00 con 0x7fd3cc108a60 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d3bb2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc112e00 con 0x7fd3cc1031c0 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d214f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d194e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3cc1031c0 0x7fd3cc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d214f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55296/0 (socket says 172.21.15.112:55296) 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3cc108a60 0x7fd3cc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.227+0000 7fd3d214f640 1 -- 172.21.15.112:0/2206752936 learned_addr learned my addr 172.21.15.112:0/2206752936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d214f640 1 -- 172.21.15.112:0/2206752936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3cc1031c0 msgr2=0x7fd3cc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d214f640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3cc1031c0 0x7fd3cc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d214f640 1 -- 172.21.15.112:0/2206752936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3cc108a60 msgr2=0x7fd3cc199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d214f640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3cc108a60 0x7fd3cc199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d214f640 1 -- 172.21.15.112:0/2206752936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc19e560 con 0x7fd3cc108f00 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d214f640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc19df20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400eca0 tx=0x7fd3c400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3c2ffd640 1 -- 172.21.15.112:0/2206752936 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4004b20 con 0x7fd3cc108f00 2024-04-25T16:40:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc19e850 con 0x7fd3cc108f00 2024-04-25T16:40:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3c2ffd640 1 -- 172.21.15.112:0/2206752936 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c4004cc0 con 0x7fd3cc108f00 2024-04-25T16:40:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3c2ffd640 1 -- 172.21.15.112:0/2206752936 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c4017670 con 0x7fd3cc108f00 2024-04-25T16:40:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.228+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc06a3c0 con 0x7fd3cc108f00 2024-04-25T16:40:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.231+0000 7fd3c2ffd640 1 -- 172.21.15.112:0/2206752936 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fd3c4005020 con 0x7fd3cc108f00 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.231+0000 7fd3daa64740 1 Processor -- start 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.231+0000 7fd3daa64740 1 -- start start 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de5c1b0 0x56378e1fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e1fc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de3b4d0 0x56378e1fc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3daa64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56378e162460 con 0x56378de332f0 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3daa64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56378e11c590 con 0x56378de3b4d0 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3daa64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56378e15e420 con 0x56378de5c1b0 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d194e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e1fc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de3b4d0 0x56378e1fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d194e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e1fc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55302/0 (socket says 172.21.15.112:55302) 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d194e640 1 -- 172.21.15.112:0/3349265999 learned_addr learned my addr 172.21.15.112:0/3349265999 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de5c1b0 0x56378e1fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3c2ffd640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd3b80afc50 0x7fd3b80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 -- 172.21.15.112:0/3349265999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de5c1b0 msgr2=0x56378e1fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de5c1b0 0x56378e1fbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 -- 172.21.15.112:0/3349265999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 msgr2=0x56378e1fc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e1fc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 -- 172.21.15.112:0/3349265999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378e1290d0 con 0x56378de3b4d0 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d214f640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd3b80afc50 0x7fd3b80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3c2ffd640 1 -- 172.21.15.112:0/2206752936 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fd3c40e0b00 con 0x7fd3cc108f00 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.232+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de3b4d0 0x56378e1fc570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc02a930 tx=0x7fd3bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0057fb2d1f2968d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.233+0000 7fd3b7fff640 1 -- 172.21.15.112:0/3349265999 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc035070 con 0x56378de3b4d0 2024-04-25T16:40:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.233+0000 7fd3b7fff640 1 -- 172.21.15.112:0/3349265999 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3bc0318a0 con 0x56378de3b4d0 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.233+0000 7fd3b7fff640 1 -- 172.21.15.112:0/3349265999 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc005280 con 0x56378de3b4d0 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.233+0000 7fd3daa64740 1 -- 172.21.15.112:0/3349265999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de3b4d0 msgr2=0x56378e1fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.233+0000 7fd3d214f640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd3b80afc50 0x7fd3b80b2160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400bb90 tx=0x7fd3c409d620 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.233+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de3b4d0 0x56378e1fc570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc02a930 tx=0x7fd3bc000bc0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 -- 172.21.15.112:0/3349265999 shutdown_connections 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de3b4d0 0x56378e1fc570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e1fc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3349265999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de5c1b0 0x56378e1fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 -- 172.21.15.112:0/3349265999 >> 172.21.15.112:0/3349265999 conn(0x56378de50300 msgr2=0x56378e1f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 -- 172.21.15.112:0/3349265999 shutdown_connections 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 -- 172.21.15.112:0/3349265999 wait complete. 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 -- start start 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.234+0000 7fd3daa64740 1 Processor -- start 2024-04-25T16:40:16.238 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e207b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e208040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e205ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56378e208bc0 con 0x56378de5c1b0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56378e0e9b60 con 0x56378de332f0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56378e208d40 con 0x56378de3b4d0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 10 client.0 _ll_register_callbacks cb 0x56378e10e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e207b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e205ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d194e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e208040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e205ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55310/0 (socket says 172.21.15.112:55310) 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 -- 172.21.15.112:0/4251172147 learned_addr learned my addr 172.21.15.112:0/4251172147 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d194e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e208040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:33128/0 (socket says 172.21.15.112:33128) 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 -- 172.21.15.112:0/4251172147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 msgr2=0x56378e208040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.239 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e208040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 -- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 msgr2=0x56378e207b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e207b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 -- 172.21.15.112:0/4251172147 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378de74210 con 0x56378de5c1b0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d194e640 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e208040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3d114d640 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e205ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x56378e205850 tx=0x7fd3bc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68220a8b8566231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3b57fa640 1 -- 172.21.15.112:0/4251172147 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc035050 con 0x56378de5c1b0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3b57fa640 1 -- 172.21.15.112:0/4251172147 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fd3bc005230 con 0x56378de5c1b0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.235+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3bc0318a0 con 0x56378de5c1b0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3b57fa640 1 -- 172.21.15.112:0/4251172147 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd3bc005430 con 0x56378de5c1b0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 10 client.15537 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56378df40620 con 0x56378de5c1b0 2024-04-25T16:40:16.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3b57fa640 1 -- 172.21.15.112:0/4251172147 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3bc0318a0 con 0x56378de5c1b0 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3b57fa640 1 -- 172.21.15.112:0/4251172147 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd3bc03d660 con 0x56378de5c1b0 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 10 client.15537 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 20 client.15537 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378de1ad80 0x56378e218550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 4 client.15537 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56378e170070 con 0x56378de1ad80 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.236+0000 7fd3d194e640 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378de1ad80 0x56378e218550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.237+0000 7fd3b57fa640 10 client.15537 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:16.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.237+0000 7fd3d194e640 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378de1ad80 0x56378e218550 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=941d89ee6e214dea server_cookie=6834d26c26f3ca5a in_seq=0 out_seq=0 2024-04-25T16:40:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3b57fa640 1 -- 172.21.15.112:0/4251172147 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56378e170070 con 0x56378de1ad80 2024-04-25T16:40:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3b57fa640 10 client.15537 handle_command_reply: tid=0 2024-04-25T16:40:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 client.15537 shutdown 2024-04-25T16:40:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 msgr2=0x56378e205ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e205ff0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x56378e205850 tx=0x7fd3bc031400 comp rx=0 tx=0).stop 2024-04-25T16:40:16.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 shutdown_connections 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378de1ad80 0x56378e218550 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e208040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e205ff0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 --2- 172.21.15.112:0/4251172147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e207b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3b57fa640 0 client.15537 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.238+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 >> 172.21.15.112:0/4251172147 conn(0x56378de50300 msgr2=0x56378e1fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.239+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 shutdown_connections 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.239+0000 7fd3daa64740 1 -- 172.21.15.112:0/4251172147 wait complete. 2024-04-25T16:40:16.243 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.239+0000 7fd3daa64740 20 client.15537 trim_cache size 0 max 16384 2024-04-25T16:40:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.254+0000 7fd3daa64740 1 Processor -- start 2024-04-25T16:40:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 -- start start 2024-04-25T16:40:16.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e1fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e21bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1b00d0 con 0x56378de5c1b0 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1ad9a0 con 0x56378de332f0 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3daa64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1ab840 con 0x56378de3b4d0 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e21bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d194e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e21bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55320/0 (socket says 172.21.15.112:55320) 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 -- 172.21.15.112:0/1135648657 learned_addr learned my addr 172.21.15.112:0/1135648657 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d194e640 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:33132/0 (socket says 172.21.15.112:33132) 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e1fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 -- 172.21.15.112:0/1135648657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 msgr2=0x56378e21b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 -- 172.21.15.112:0/1135648657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 msgr2=0x56378e1fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e1fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d114d640 1 -- 172.21.15.112:0/1135648657 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378e1fd340 con 0x56378de5c1b0 2024-04-25T16:40:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.255+0000 7fd3d194e640 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.256+0000 7fd3d114d640 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e21bac0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc02ac00 tx=0x7fd3bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d807c9086fda2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.256+0000 7fd3b77fe640 1 -- 172.21.15.112:0/1135648657 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc035040 con 0x56378de5c1b0 2024-04-25T16:40:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.256+0000 7fd3b77fe640 1 -- 172.21.15.112:0/1135648657 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3bc03d660 con 0x56378de5c1b0 2024-04-25T16:40:16.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.256+0000 7fd3b77fe640 1 -- 172.21.15.112:0/1135648657 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc005200 con 0x56378de5c1b0 2024-04-25T16:40:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 -- 172.21.15.112:0/1135648657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 msgr2=0x56378e21bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e21bac0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc02ac00 tx=0x7fd3bc000bc0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 -- 172.21.15.112:0/1135648657 shutdown_connections 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de5c1b0 0x56378e21bac0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 --2- 172.21.15.112:0/1135648657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de332f0 0x56378e1fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 -- 172.21.15.112:0/1135648657 >> 172.21.15.112:0/1135648657 conn(0x56378de50300 msgr2=0x56378e1fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 -- 172.21.15.112:0/1135648657 shutdown_connections 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 -- 172.21.15.112:0/1135648657 wait complete. 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 -- start start 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 Processor -- start 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.257+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e21a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de5c1b0 0x56378e27e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.260 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1abeb0 con 0x56378de332f0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1ad2d0 con 0x56378de5c1b0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc1ac530 con 0x56378de3b4d0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 10 client.0 _ll_register_callbacks cb 0x7fd3c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e21a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e21a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:55334/0 (socket says 172.21.15.112:55334) 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 -- 172.21.15.112:0/3949673066 learned_addr learned my addr 172.21.15.112:0/3949673066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d194e640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de5c1b0 0x56378e27e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 -- 172.21.15.112:0/3949673066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 msgr2=0x56378e21aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 -- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de5c1b0 msgr2=0x56378e27e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de5c1b0 0x56378e27e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 -- 172.21.15.112:0/3949673066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56378e27ebf0 con 0x56378de332f0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3d114d640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e21a970 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc005f70 tx=0x7fd3bc004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bd7d181dcda4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:16.261 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3b57fa640 1 -- 172.21.15.112:0/3949673066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc035030 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3bc03d660 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3b57fa640 1 -- 172.21.15.112:0/3949673066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fd3bc0046c0 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3b57fa640 1 -- 172.21.15.112:0/3949673066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fd3bc0048c0 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 10 client.15543 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.258+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56378e27ebf0 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3b57fa640 1 -- 172.21.15.112:0/3949673066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3bc03d660 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3b57fa640 1 -- 172.21.15.112:0/3949673066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fd3bc06fc60 con 0x56378de332f0 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3daa64740 10 client.15543 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3daa64740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3daa64740 20 client.15543 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378ddf01c0 0x56378e2820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:16.262 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3daa64740 4 client.15543 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:40:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56378e169ee0 con 0x56378ddf01c0 2024-04-25T16:40:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3d194e640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378ddf01c0 0x56378e2820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.259+0000 7fd3b57fa640 10 client.15543 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:16.263 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.260+0000 7fd3d194e640 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378ddf01c0 0x56378e2820e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5161c7c70f7fee99 server_cookie=fde343b2af5a014e in_seq=0 out_seq=0 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.423+0000 7fd3b57fa640 1 -- 172.21.15.112:0/3949673066 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56378e169ee0 con 0x56378ddf01c0 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.423+0000 7fd3b57fa640 10 client.15543 handle_command_reply: tid=0 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.423+0000 7fd3daa64740 1 client.15543 shutdown 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 msgr2=0x56378e21a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e21a970 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc005f70 tx=0x7fd3bc004ee0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 shutdown_connections 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56378ddf01c0 0x56378e2820e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56378de3b4d0 0x56378e21aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56378de5c1b0 0x56378e27e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 --2- 172.21.15.112:0/3949673066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56378de332f0 0x56378e21a970 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3b57fa640 0 client.15543 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 >> 172.21.15.112:0/3949673066 conn(0x56378de50300 msgr2=0x56378e205850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 shutdown_connections 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 1 -- 172.21.15.112:0/3949673066 wait complete. 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.424+0000 7fd3daa64740 20 client.15543 trim_cache size 0 max 16384 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:40:16.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:40:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd3b80afc50 msgr2=0x7fd3b80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd3b80afc50 0x7fd3b80b2160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400bb90 tx=0x7fd3c409d620 comp rx=0 tx=0).stop 2024-04-25T16:40:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 msgr2=0x7fd3cc19df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:16.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc19df20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400eca0 tx=0x7fd3c400b690 comp rx=0 tx=0).stop 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 shutdown_connections 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd3b80afc50 0x7fd3b80b2160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd3cc108f00 0x7fd3cc19df20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3cc108a60 0x7fd3cc199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 --2- 172.21.15.112:0/2206752936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3cc1031c0 0x7fd3cc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 >> 172.21.15.112:0/2206752936 conn(0x7fd3cc0781b0 msgr2=0x7fd3cc104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 shutdown_connections 2024-04-25T16:40:16.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:16.425+0000 7fd3d3bb2640 1 -- 172.21.15.112:0/2206752936 wait complete. 2024-04-25T16:40:16.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 59693 15.43 MB/sec warmup 22 sec latency 86.402 ms 2024-04-25T16:40:16.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:16 smithi133 ceph-mon[32691]: pgmap v57: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-25T16:40:17.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:16 smithi116 ceph-mon[32712]: pgmap v57: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-25T16:40:17.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:16 smithi112 ceph-mon[29122]: pgmap v57: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-25T16:40:17.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 62021 15.27 MB/sec warmup 23 sec latency 16.001 ms 2024-04-25T16:40:18.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 64670 15.24 MB/sec warmup 24 sec latency 13.559 ms 2024-04-25T16:40:19.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:19 smithi116 ceph-mon[32712]: pgmap v58: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-25T16:40:19.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:19 smithi112 ceph-mon[29122]: pgmap v58: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-25T16:40:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:19 smithi133 ceph-mon[32691]: pgmap v58: 97 pgs: 97 active+clean; 27 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-25T16:40:19.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 67745 15.36 MB/sec warmup 25 sec latency 20.359 ms 2024-04-25T16:40:20.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 70629 15.29 MB/sec warmup 26 sec latency 11.754 ms 2024-04-25T16:40:21.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:21 smithi112 ceph-mon[29122]: pgmap v59: 97 pgs: 97 active+clean; 22 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 393 op/s 2024-04-25T16:40:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:21 smithi133 ceph-mon[32691]: pgmap v59: 97 pgs: 97 active+clean; 22 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 393 op/s 2024-04-25T16:40:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:21 smithi116 ceph-mon[32712]: pgmap v59: 97 pgs: 97 active+clean; 22 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 393 op/s 2024-04-25T16:40:21.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 73366 15.20 MB/sec warmup 27 sec latency 28.061 ms 2024-04-25T16:40:22.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 75611 15.16 MB/sec warmup 28 sec latency 13.809 ms 2024-04-25T16:40:23.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:23 smithi112 ceph-mon[29122]: pgmap v60: 97 pgs: 97 active+clean; 22 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-04-25T16:40:23.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:23 smithi133 ceph-mon[32691]: pgmap v60: 97 pgs: 97 active+clean; 22 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-04-25T16:40:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:23 smithi116 ceph-mon[32712]: pgmap v60: 97 pgs: 97 active+clean; 22 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 1.7 MiB/s wr, 306 op/s 2024-04-25T16:40:23.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 78544 15.23 MB/sec warmup 29 sec latency 8.871 ms 2024-04-25T16:40:24.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 81631 15.31 MB/sec warmup 30 sec latency 21.546 ms 2024-04-25T16:40:25.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:25 smithi112 ceph-mon[29122]: pgmap v61: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 394 op/s 2024-04-25T16:40:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:25 smithi133 ceph-mon[32691]: pgmap v61: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 394 op/s 2024-04-25T16:40:25.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:25 smithi116 ceph-mon[32712]: pgmap v61: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 394 op/s 2024-04-25T16:40:25.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 84194 15.12 MB/sec warmup 31 sec latency 10.056 ms 2024-04-25T16:40:26.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 86050 15.02 MB/sec warmup 32 sec latency 29.540 ms 2024-04-25T16:40:27.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:27 smithi112 ceph-mon[29122]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-04-25T16:40:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:27 smithi133 ceph-mon[32691]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-04-25T16:40:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:27 smithi116 ceph-mon[32712]: pgmap v62: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-04-25T16:40:27.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 89050 15.11 MB/sec warmup 33 sec latency 9.650 ms 2024-04-25T16:40:28.867 INFO:tasks.workunit.client.0.smithi112.stdout: 1 91899 15.07 MB/sec warmup 34 sec latency 17.354 ms 2024-04-25T16:40:29.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:29 smithi112 ceph-mon[29122]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-04-25T16:40:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:29 smithi133 ceph-mon[32691]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-04-25T16:40:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:29 smithi116 ceph-mon[32712]: pgmap v63: 97 pgs: 97 active+clean; 29 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 1.5 MiB/s wr, 307 op/s 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 -- 172.21.15.112:0/3537048634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96810b6d0 msgr2=0x7fa96810daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 --2- 172.21.15.112:0/3537048634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96810b6d0 0x7fa96810daf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009860 tx=0x7fa95c02a340 comp rx=0 tx=0).stop 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 -- 172.21.15.112:0/3537048634 shutdown_connections 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 --2- 172.21.15.112:0/3537048634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96810b6d0 0x7fa96810daf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 --2- 172.21.15.112:0/3537048634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810e0c0 0x7fa9681105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 --2- 172.21.15.112:0/3537048634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa968104c70 0x7fa968105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 -- 172.21.15.112:0/3537048634 >> 172.21.15.112:0/3537048634 conn(0x7fa9680781b0 msgr2=0x7fa9680785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 -- 172.21.15.112:0/3537048634 shutdown_connections 2024-04-25T16:40:29.523 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.524+0000 7fa96c849640 1 -- 172.21.15.112:0/3537048634 wait complete. 2024-04-25T16:40:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 Processor -- start 2024-04-25T16:40:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 -- start start 2024-04-25T16:40:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 0x7fa96819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810b6d0 0x7fa96819ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96810e0c0 0x7fa96819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa968113d20 con 0x7fa96810b6d0 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa968113ba0 con 0x7fa96810e0c0 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96c849640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa968113ea0 con 0x7fa968104c70 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa965d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810b6d0 0x7fa96819ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 0x7fa96819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 0x7fa96819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:41754/0 (socket says 172.21.15.112:41754) 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa965d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810b6d0 0x7fa96819ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41064/0 (socket says 172.21.15.112:41064) 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa96659c640 1 -- 172.21.15.112:0/2149619241 learned_addr learned my addr 172.21.15.112:0/2149619241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.525+0000 7fa966d9d640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96810e0c0 0x7fa96819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96659c640 1 -- 172.21.15.112:0/2149619241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96810e0c0 msgr2=0x7fa96819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96659c640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96810e0c0 0x7fa96819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96659c640 1 -- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810b6d0 msgr2=0x7fa96819ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96659c640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810b6d0 0x7fa96819ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96659c640 1 -- 172.21.15.112:0/2149619241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96819f640 con 0x7fa968104c70 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96659c640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 0x7fa96819a7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9f0 tx=0x7fa95800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa9577fe640 1 -- 172.21.15.112:0/2149619241 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95800cd80 con 0x7fa968104c70 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96819f930 con 0x7fa968104c70 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa9577fe640 1 -- 172.21.15.112:0/2149619241 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9580054d0 con 0x7fa968104c70 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa9577fe640 1 -- 172.21.15.112:0/2149619241 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958010690 con 0x7fa968104c70 2024-04-25T16:40:29.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.526+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96819fe00 con 0x7fa968104c70 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.528+0000 7fa9577fe640 1 -- 172.21.15.112:0/2149619241 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fa958004e90 con 0x7fa968104c70 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.528+0000 7fa9737d5740 1 Processor -- start 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 -- start start 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6324af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 0x5579b6325030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f644d0 0x5579b6325570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5579b628b460 con 0x5579b5f5c2f0 2024-04-25T16:40:29.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5579b6245590 con 0x5579b5f644d0 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9737d5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5579b6287420 con 0x5579b5f851b0 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 0x5579b6325030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa966d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6324af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 0x5579b6325030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41072/0 (socket says 172.21.15.112:41072) 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 -- 172.21.15.112:0/1618314680 learned_addr learned my addr 172.21.15.112:0/1618314680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa965d9b640 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f644d0 0x5579b6325570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa9577fe640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa9500afc50 0x7fa9500b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 -- 172.21.15.112:0/1618314680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 msgr2=0x5579b6324af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6324af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 -- 172.21.15.112:0/1618314680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f644d0 msgr2=0x5579b6325570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f644d0 0x5579b6325570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa96659c640 1 -- 172.21.15.112:0/1618314680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b62520d0 con 0x5579b5f5c2f0 2024-04-25T16:40:29.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa965d9b640 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f644d0 0x5579b6325570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.529+0000 7fa966d9d640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa9500afc50 0x7fa9500b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.530+0000 7fa9577fe640 1 -- 172.21.15.112:0/2149619241 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fa9580df470 con 0x7fa968104c70 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.530+0000 7fa96659c640 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 0x5579b6325030 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa9580dcb10 tx=0x7fa95806db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41398013472b91f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.530+0000 7fa954ff9640 1 -- 172.21.15.112:0/1618314680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580a73c0 con 0x5579b5f5c2f0 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.530+0000 7fa954ff9640 1 -- 172.21.15.112:0/1618314680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9580a7100 con 0x5579b5f5c2f0 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.530+0000 7fa954ff9640 1 -- 172.21.15.112:0/1618314680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958005180 con 0x5579b5f5c2f0 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 -- 172.21.15.112:0/1618314680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 msgr2=0x5579b6325030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa966d9d640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa9500afc50 0x7fa9500b2160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa960009890 tx=0x7fa960004a40 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 0x5579b6325030 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa9580dcb10 tx=0x7fa95806db80 comp rx=0 tx=0).stop 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 -- 172.21.15.112:0/1618314680 shutdown_connections 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f644d0 0x5579b6325570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f5c2f0 0x5579b6325030 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 --2- 172.21.15.112:0/1618314680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6324af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 -- 172.21.15.112:0/1618314680 >> 172.21.15.112:0/1618314680 conn(0x5579b5f79300 msgr2=0x5579b63212a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:29.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 -- 172.21.15.112:0/1618314680 shutdown_connections 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 -- 172.21.15.112:0/1618314680 wait complete. 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 -- start start 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.531+0000 7fa9737d5740 1 Processor -- start 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b632f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6335010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5579b6331b60 con 0x5579b5f644d0 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5579b6212b60 con 0x5579b5f5c2f0 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5579b6331ce0 con 0x5579b5f851b0 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 10 client.0 _ll_register_callbacks cb 0x5579b62372b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa965d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b632f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa965d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41076/0 (socket says 172.21.15.112:41076) 2024-04-25T16:40:29.537 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa965d9b640 1 -- 172.21.15.112:0/3518220049 learned_addr learned my addr 172.21.15.112:0/3518220049 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa966d9d640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6335010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 -- 172.21.15.112:0/3518220049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 msgr2=0x5579b6335010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6335010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 -- 172.21.15.112:0/3518220049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 msgr2=0x5579b632f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 -- 172.21.15.112:0/3518220049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b5f9d210 con 0x5579b5f5c2f0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa965d9b640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa96659c640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b632f120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9580dcc00 tx=0x7fa9580742e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59ad894c396d3833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa94dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa94dffb640 1 -- 172.21.15.112:0/3518220049 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580a6d60 con 0x5579b5f5c2f0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9580a7100 con 0x5579b5f5c2f0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.532+0000 7fa94dffb640 1 -- 172.21.15.112:0/3518220049 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fa9580affc0 con 0x5579b5f5c2f0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa94dffb640 1 -- 172.21.15.112:0/3518220049 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa9580a6f40 con 0x5579b5f5c2f0 2024-04-25T16:40:29.538 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 10 client.25253 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5579b6069620 con 0x5579b5f5c2f0 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa94dffb640 1 -- 172.21.15.112:0/3518220049 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9580a7100 con 0x5579b5f5c2f0 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa94dffb640 1 -- 172.21.15.112:0/3518220049 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa9580de060 con 0x5579b5f5c2f0 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 10 client.25253 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 10 client.25253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 20 client.25253 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f43d80 0x5579b6341430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 4 client.25253 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579b6299070 con 0x5579b5f43d80 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.533+0000 7fa965d9b640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f43d80 0x5579b6341430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.534+0000 7fa94dffb640 10 client.25253 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.534+0000 7fa965d9b640 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f43d80 0x5579b6341430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc21650e187aea1 server_cookie=3fdea1f1633a291f in_seq=0 out_seq=0 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa94dffb640 1 -- 172.21.15.112:0/3518220049 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5579b6299070 con 0x5579b5f43d80 2024-04-25T16:40:29.539 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa94dffb640 10 client.25253 handle_command_reply: tid=0 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 client.25253 shutdown 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 msgr2=0x5579b632f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b632f120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9580dcc00 tx=0x7fa9580742e0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 shutdown_connections 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f43d80 0x5579b6341430 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f851b0 0x5579b6335010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa94dffb640 0 client.25253 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3518220049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b632f120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 >> 172.21.15.112:0/3518220049 conn(0x5579b5f79300 msgr2=0x5579b6324920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 shutdown_connections 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 1 -- 172.21.15.112:0/3518220049 wait complete. 2024-04-25T16:40:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.535+0000 7fa9737d5740 20 client.25253 trim_cache size 0 max 16384 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.550+0000 7fa9737d5740 1 Processor -- start 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.550+0000 7fa9737d5740 1 -- start start 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f43d80 0x5579b6330d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b6331280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa9737d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681b23b0 con 0x5579b5f644d0 2024-04-25T16:40:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa9737d5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681afc80 con 0x5579b5f5c2f0 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa9737d5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681adad0 con 0x5579b5f43d80 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa966d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa966d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41092/0 (socket says 172.21.15.112:41092) 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa966d9d640 1 -- 172.21.15.112:0/2933488528 learned_addr learned my addr 172.21.15.112:0/2933488528 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f43d80 0x5579b6330d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa965d9b640 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b6331280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 -- 172.21.15.112:0/2933488528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 msgr2=0x5579b6331280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b6331280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 -- 172.21.15.112:0/2933488528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 msgr2=0x5579b632c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 -- 172.21.15.112:0/2933488528 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b6328e40 con 0x5579b5f43d80 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa966d9d640 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa96659c640 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f43d80 0x5579b6330d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9580a7580 tx=0x7fa9580901d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f3c59e2097b556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa94ffff640 1 -- 172.21.15.112:0/2933488528 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580a6d60 con 0x5579b5f43d80 2024-04-25T16:40:29.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.551+0000 7fa94ffff640 1 -- 172.21.15.112:0/2933488528 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9580dd020 con 0x5579b5f43d80 2024-04-25T16:40:29.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa94ffff640 1 -- 172.21.15.112:0/2933488528 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958005180 con 0x5579b5f43d80 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 -- 172.21.15.112:0/2933488528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f43d80 msgr2=0x5579b6330d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f43d80 0x5579b6330d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9580a7580 tx=0x7fa9580901d0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 -- 172.21.15.112:0/2933488528 shutdown_connections 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f644d0 0x5579b632c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b6331280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 --2- 172.21.15.112:0/2933488528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f43d80 0x5579b6330d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 -- 172.21.15.112:0/2933488528 >> 172.21.15.112:0/2933488528 conn(0x5579b5f79300 msgr2=0x5579b6330630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 -- 172.21.15.112:0/2933488528 shutdown_connections 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.552+0000 7fa9737d5740 1 -- 172.21.15.112:0/2933488528 wait complete. 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 -- start start 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 Processor -- start 2024-04-25T16:40:29.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 0x5579b633ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b633b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f644d0 0x5579b63a7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ae140 con 0x5579b5f43d80 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681af5b0 con 0x5579b5f5c2f0 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ae810 con 0x5579b5f644d0 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa9737d5740 10 client.0 _ll_register_callbacks cb 0x7fa960009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa96659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 0x5579b633ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa96659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 0x5579b633ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41102/0 (socket says 172.21.15.112:41102) 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa965d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b633b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa96659c640 1 -- 172.21.15.112:0/3959236379 learned_addr learned my addr 172.21.15.112:0/3959236379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa966d9d640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f644d0 0x5579b63a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa965d9b640 1 -- 172.21.15.112:0/3959236379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f644d0 msgr2=0x5579b63a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa965d9b640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f644d0 0x5579b63a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa965d9b640 1 -- 172.21.15.112:0/3959236379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 msgr2=0x5579b633ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:29.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa965d9b640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 0x5579b633ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.553+0000 7fa965d9b640 1 -- 172.21.15.112:0/3959236379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b63a7aa0 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa966d9d640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f644d0 0x5579b63a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa96659c640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 0x5579b633ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa965d9b640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b633b240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa95c000c00 tx=0x7fa95c004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a704283167c7ab51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa94dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa94dffb640 1 -- 172.21.15.112:0/3959236379 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c04e060 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa94dffb640 1 -- 172.21.15.112:0/3959236379 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7fa95c030650 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9580dd020 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa94dffb640 1 -- 172.21.15.112:0/3959236379 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fa95c037040 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa9737d5740 10 client.25259 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5579b63a8d60 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa94dffb640 1 -- 172.21.15.112:0/3959236379 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9580dd020 con 0x5579b5f5c2f0 2024-04-25T16:40:29.558 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.554+0000 7fa94dffb640 1 -- 172.21.15.112:0/3959236379 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7fa95c049510 con 0x5579b5f5c2f0 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa9737d5740 10 client.25259 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa9737d5740 10 client.25259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa9737d5740 20 client.25259 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f851b0 0x5579b63ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa9737d5740 4 client.25259 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579b6292ee0 con 0x5579b5f851b0 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa96659c640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f851b0 0x5579b63ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa94dffb640 10 client.25259 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:29.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.555+0000 7fa96659c640 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f851b0 0x5579b63ab500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567e0eca8203455a server_cookie=9d22548afd31fde7 in_seq=0 out_seq=0 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.690+0000 7fa94dffb640 1 -- 172.21.15.112:0/3959236379 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579b6292ee0 con 0x5579b5f851b0 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.690+0000 7fa94dffb640 10 client.25259 handle_command_reply: tid=0 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 client.25259 shutdown 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 msgr2=0x5579b633b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b633b240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa95c000c00 tx=0x7fa95c004500 comp rx=0 tx=0).stop 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 shutdown_connections 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5579b5f851b0 0x5579b63ab500 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5579b5f644d0 0x5579b63a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5579b5f5c2f0 0x5579b633b240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa94dffb640 0 client.25259 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 --2- 172.21.15.112:0/3959236379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5579b5f43d80 0x5579b633ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 >> 172.21.15.112:0/3959236379 conn(0x5579b5f79300 msgr2=0x5579b6330fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 shutdown_connections 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 1 -- 172.21.15.112:0/3959236379 wait complete. 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.691+0000 7fa9737d5740 20 client.25259 trim_cache size 0 max 16384 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:40:29.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:40:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa9500afc50 msgr2=0x7fa9500b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa9500afc50 0x7fa9500b2160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa960009890 tx=0x7fa960004a40 comp rx=0 tx=0).stop 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 msgr2=0x7fa96819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 0x7fa96819a7e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9f0 tx=0x7fa95800edf0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 shutdown_connections 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa9500afc50 0x7fa9500b2160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa96810e0c0 0x7fa96819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa96810b6d0 0x7fa96819ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 --2- 172.21.15.112:0/2149619241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa968104c70 0x7fa96819a7e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 >> 172.21.15.112:0/2149619241 conn(0x7fa9680781b0 msgr2=0x7fa968101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 shutdown_connections 2024-04-25T16:40:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:29.692+0000 7fa96c849640 1 -- 172.21.15.112:0/2149619241 wait complete. 2024-04-25T16:40:29.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 93342 14.86 MB/sec warmup 35 sec latency 55.887 ms 2024-04-25T16:40:30.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 96242 14.93 MB/sec warmup 36 sec latency 8.385 ms 2024-04-25T16:40:31.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:31 smithi116 ceph-mon[32712]: pgmap v64: 97 pgs: 97 active+clean; 54 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-04-25T16:40:31.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:31 smithi112 ceph-mon[29122]: pgmap v64: 97 pgs: 97 active+clean; 54 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-04-25T16:40:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:31 smithi133 ceph-mon[32691]: pgmap v64: 97 pgs: 97 active+clean; 54 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 402 op/s 2024-04-25T16:40:31.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 98749 14.86 MB/sec warmup 37 sec latency 82.287 ms 2024-04-25T16:40:32.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 100892 14.78 MB/sec warmup 38 sec latency 14.634 ms 2024-04-25T16:40:33.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:33 smithi112 ceph-mon[29122]: pgmap v65: 97 pgs: 97 active+clean; 54 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 182 op/s 2024-04-25T16:40:33.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:33 smithi133 ceph-mon[32691]: pgmap v65: 97 pgs: 97 active+clean; 54 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 182 op/s 2024-04-25T16:40:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:33 smithi116 ceph-mon[32712]: pgmap v65: 97 pgs: 97 active+clean; 54 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 182 op/s 2024-04-25T16:40:33.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 103483 14.76 MB/sec warmup 39 sec latency 8.687 ms 2024-04-25T16:40:34.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:34 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:40:34.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:34 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:40:34.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:34 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:40:34.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 106450 14.83 MB/sec warmup 40 sec latency 19.053 ms 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: pgmap v66: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:40:35.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:40:35.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: pgmap v66: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-04-25T16:40:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:40:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:40:35.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: pgmap v66: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:40:35.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:40:35.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 108650 14.65 MB/sec warmup 41 sec latency 19.646 ms 2024-04-25T16:40:36.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 110488 14.61 MB/sec warmup 42 sec latency 51.408 ms 2024-04-25T16:40:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:37 smithi112 ceph-mon[29122]: pgmap v67: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-04-25T16:40:37.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:37 smithi133 ceph-mon[32691]: pgmap v67: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-04-25T16:40:37.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:37 smithi116 ceph-mon[32712]: pgmap v67: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-04-25T16:40:37.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 113613 14.69 MB/sec warmup 43 sec latency 13.715 ms 2024-04-25T16:40:38.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 115912 14.54 MB/sec warmup 44 sec latency 12.564 ms 2024-04-25T16:40:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:39 smithi112 ceph-mon[29122]: pgmap v68: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-04-25T16:40:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:39 smithi133 ceph-mon[32691]: pgmap v68: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-04-25T16:40:39.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:39 smithi116 ceph-mon[32712]: pgmap v68: 97 pgs: 97 active+clean; 46 MiB data, 493 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 315 op/s 2024-04-25T16:40:39.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 118068 14.51 MB/sec warmup 45 sec latency 19.487 ms 2024-04-25T16:40:40.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 121025 14.59 MB/sec warmup 46 sec latency 8.364 ms 2024-04-25T16:40:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:41 smithi112 ceph-mon[29122]: pgmap v69: 97 pgs: 97 active+clean; 51 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-04-25T16:40:41.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:41 smithi133 ceph-mon[32691]: pgmap v69: 97 pgs: 97 active+clean; 51 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-04-25T16:40:41.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:41 smithi116 ceph-mon[32712]: pgmap v69: 97 pgs: 97 active+clean; 51 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-04-25T16:40:41.868 INFO:tasks.workunit.client.0.smithi112.stdout: 1 123191 14.45 MB/sec warmup 47 sec latency 101.358 ms 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 -- 172.21.15.112:0/4275125431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 msgr2=0x7f91dc102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 --2- 172.21.15.112:0/4275125431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc102610 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009900 tx=0x7f91d802a410 comp rx=0 tx=0).stop 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 -- 172.21.15.112:0/4275125431 shutdown_connections 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 --2- 172.21.15.112:0/4275125431 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91dc1031c0 0x7f91dc10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 --2- 172.21.15.112:0/4275125431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc108f00 0x7f91dc102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 --2- 172.21.15.112:0/4275125431 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc102610 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 -- 172.21.15.112:0/4275125431 >> 172.21.15.112:0/4275125431 conn(0x7f91dc0781b0 msgr2=0x7f91dc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 -- 172.21.15.112:0/4275125431 shutdown_connections 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.793+0000 7f91e4413640 1 -- 172.21.15.112:0/4275125431 wait complete. 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 Processor -- start 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 -- start start 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc1031c0 0x7f91dc1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc1a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91dc108f00 0x7f91dc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc115f70 con 0x7f91dc108a60 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc115df0 con 0x7f91dc1031c0 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e4413640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1160f0 con 0x7f91dc108f00 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e19ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc1a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e19ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc1a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49000/0 (socket says 172.21.15.112:49000) 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e19ae640 1 -- 172.21.15.112:0/3061663233 learned_addr learned my addr 172.21.15.112:0/3061663233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.794+0000 7f91e21af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc1031c0 0x7f91dc1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e29b0640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91dc108f00 0x7f91dc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e19ae640 1 -- 172.21.15.112:0/3061663233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91dc108f00 msgr2=0x7f91dc19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e19ae640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91dc108f00 0x7f91dc19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e19ae640 1 -- 172.21.15.112:0/3061663233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc1031c0 msgr2=0x7f91dc1a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e19ae640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc1031c0 0x7f91dc1a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e19ae640 1 -- 172.21.15.112:0/3061663233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc19ce90 con 0x7f91dc108a60 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e21af640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc1031c0 0x7f91dc1a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e19ae640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc1a2ac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00e9f0 tx=0x7f91cc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91d37fe640 1 -- 172.21.15.112:0/3061663233 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00ccc0 con 0x7f91dc108a60 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc19d180 con 0x7f91dc108a60 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc1a93d0 con 0x7f91dc108a60 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91d37fe640 1 -- 172.21.15.112:0/3061663233 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91cc0035f0 con 0x7f91dc108a60 2024-04-25T16:40:42.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.795+0000 7f91d37fe640 1 -- 172.21.15.112:0/3061663233 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00ce60 con 0x7f91dc108a60 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91d37fe640 1 -- 172.21.15.112:0/3061663233 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f91cc010690 con 0x7f91dc108a60 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 Processor -- start 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 -- start start 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 0x558009eb5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009aed2f0 0x558009eb6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009af54d0 0x558009eb6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558009e1c460 con 0x558009b161b0 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558009dd6590 con 0x558009aed2f0 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91eb39f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558009e18420 con 0x558009af54d0 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.798+0000 7f91e29b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 0x558009eb5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e21af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009aed2f0 0x558009eb6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 0x558009eb5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49002/0 (socket says 172.21.15.112:49002) 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 -- 172.21.15.112:0/2507571161 learned_addr learned my addr 172.21.15.112:0/2507571161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e19ae640 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009af54d0 0x558009eb6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91d37fe640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f91c80afcd0 0x7f91c80b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 -- 172.21.15.112:0/2507571161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009af54d0 msgr2=0x558009eb6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009af54d0 0x558009eb6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 -- 172.21.15.112:0/2507571161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009aed2f0 msgr2=0x558009eb6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009aed2f0 0x558009eb6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 -- 172.21.15.112:0/2507571161 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558009de30d0 con 0x558009b161b0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e21af640 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009aed2f0 0x558009eb6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91d37fe640 1 -- 172.21.15.112:0/3061663233 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f91cc0df2f0 con 0x7f91dc108a60 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f91c80afcd0 0x7f91c80b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91e29b0640 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 0x558009eb5af0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f91d4011590 tx=0x7f91d4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca63a9b700ab13fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91d0bfe640 1 -- 172.21.15.112:0/2507571161 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4017050 con 0x558009b161b0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91d0bfe640 1 -- 172.21.15.112:0/2507571161 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d4004750 con 0x558009b161b0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.799+0000 7f91d0bfe640 1 -- 172.21.15.112:0/2507571161 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4025ad0 con 0x558009b161b0 2024-04-25T16:40:42.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 -- 172.21.15.112:0/2507571161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 msgr2=0x558009eb5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91e29b0640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f91c80afcd0 0x7f91c80b21e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91d4020000 tx=0x7f91d4021000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 0x558009eb5af0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f91d4011590 tx=0x7f91d4011a50 comp rx=0 tx=0).stop 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 -- 172.21.15.112:0/2507571161 shutdown_connections 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009af54d0 0x558009eb6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009aed2f0 0x558009eb6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2507571161 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009b161b0 0x558009eb5af0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 -- 172.21.15.112:0/2507571161 >> 172.21.15.112:0/2507571161 conn(0x558009b0a300 msgr2=0x558009eb22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 -- 172.21.15.112:0/2507571161 shutdown_connections 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.800+0000 7f91eb39f740 1 -- 172.21.15.112:0/2507571161 wait complete. 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 -- start start 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 Processor -- start 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009ec1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009b161b0 0x558009ebffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558009ec2c20 con 0x558009aed2f0 2024-04-25T16:40:42.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558009da3b60 con 0x558009af54d0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558009ec2da0 con 0x558009b161b0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91e21af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009ec1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91eb39f740 10 client.0 _ll_register_callbacks cb 0x558009dc82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91e19ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91e21af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009ec1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49004/0 (socket says 172.21.15.112:49004) 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91e21af640 1 -- 172.21.15.112:0/2858354749 learned_addr learned my addr 172.21.15.112:0/2858354749 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.801+0000 7f91e29b0640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009b161b0 0x558009ebffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e19ae640 1 -- 172.21.15.112:0/2858354749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009b161b0 msgr2=0x558009ebffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e19ae640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009b161b0 0x558009ebffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e19ae640 1 -- 172.21.15.112:0/2858354749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 msgr2=0x558009ec1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e19ae640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009ec1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e19ae640 1 -- 172.21.15.112:0/2858354749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558009b2e210 con 0x558009af54d0 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e21af640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009ec1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:42.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91e19ae640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec2050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0dca20 tx=0x7f91cc0a3460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2aa337f3d8fe5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91c5ffb640 1 -- 172.21.15.112:0/2858354749 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0de070 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91c5ffb640 1 -- 172.21.15.112:0/2858354749 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f91cc0b0040 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d4004750 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91c5ffb640 1 -- 172.21.15.112:0/2858354749 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f91cc0ae010 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91eb39f740 10 client.25271 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558009bfa620 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.802+0000 7f91c5ffb640 1 -- 172.21.15.112:0/2858354749 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d4004750 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91c5ffb640 1 -- 172.21.15.112:0/2858354749 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f91cc0aeba0 con 0x558009af54d0 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91eb39f740 10 client.25271 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91eb39f740 10 client.25271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91eb39f740 20 client.25271 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009ad4d80 0x558009ed27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91eb39f740 4 client.25271 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:40:42.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558009e2a070 con 0x558009ad4d80 2024-04-25T16:40:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91e21af640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009ad4d80 0x558009ed27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91c5ffb640 10 client.25271 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:42.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.803+0000 7f91e21af640 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009ad4d80 0x558009ed27a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf35a5ce595dfe3a server_cookie=9b8225770ccd2474 in_seq=0 out_seq=0 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91c5ffb640 1 -- 172.21.15.112:0/2858354749 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558009e2a070 con 0x558009ad4d80 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91c5ffb640 10 client.25271 handle_command_reply: tid=0 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 client.25271 shutdown 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 msgr2=0x558009ec2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec2050 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0dca20 tx=0x7f91cc0a3460 comp rx=0 tx=0).stop 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 shutdown_connections 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009ad4d80 0x558009ed27a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009b161b0 0x558009ebffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec2050 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91c5ffb640 0 client.25271 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 --2- 172.21.15.112:0/2858354749 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009ec1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 >> 172.21.15.112:0/2858354749 conn(0x558009b0a300 msgr2=0x558009eb5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 shutdown_connections 2024-04-25T16:40:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 1 -- 172.21.15.112:0/2858354749 wait complete. 2024-04-25T16:40:42.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.804+0000 7f91eb39f740 20 client.25271 trim_cache size 0 max 16384 2024-04-25T16:40:42.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 Processor -- start 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 -- start start 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009ad4d80 0x558009ebe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009aed2f0 0x558009ebedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 0x558009ed7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b4630 con 0x558009af54d0 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b1f00 con 0x558009ad4d80 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91eb39f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1afd50 con 0x558009aed2f0 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91e29b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 0x558009ed7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91e29b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 0x558009ed7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49012/0 (socket says 172.21.15.112:49012) 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91e29b0640 1 -- 172.21.15.112:0/1524591067 learned_addr learned my addr 172.21.15.112:0/1524591067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.820+0000 7f91e21af640 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009ad4d80 0x558009ebe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e19ae640 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009aed2f0 0x558009ebedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e29b0640 1 -- 172.21.15.112:0/1524591067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009aed2f0 msgr2=0x558009ebedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e29b0640 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009aed2f0 0x558009ebedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e29b0640 1 -- 172.21.15.112:0/1524591067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009ad4d80 msgr2=0x558009ebe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e29b0640 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009ad4d80 0x558009ebe880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e29b0640 1 -- 172.21.15.112:0/1524591067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558009ebbb70 con 0x558009af54d0 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91e29b0640 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 0x558009ed7fa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f91d4025760 tx=0x7f91d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532e6a2b4bc04eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91c7fff640 1 -- 172.21.15.112:0/1524591067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4034020 con 0x558009af54d0 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91c7fff640 1 -- 172.21.15.112:0/1524591067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d402b040 con 0x558009af54d0 2024-04-25T16:40:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.821+0000 7f91c7fff640 1 -- 172.21.15.112:0/1524591067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d402a030 con 0x558009af54d0 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- 172.21.15.112:0/1524591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 msgr2=0x558009ed7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 0x558009ed7fa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f91d4025760 tx=0x7f91d4014000 comp rx=0 tx=0).stop 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- 172.21.15.112:0/1524591067 shutdown_connections 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009aed2f0 0x558009ebedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009af54d0 0x558009ed7fa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1524591067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009ad4d80 0x558009ebe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- 172.21.15.112:0/1524591067 >> 172.21.15.112:0/1524591067 conn(0x558009b0a300 msgr2=0x558009eb97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- 172.21.15.112:0/1524591067 shutdown_connections 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- 172.21.15.112:0/1524591067 wait complete. 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- start start 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 Processor -- start 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009ad4d80 0x558009f33510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009f33a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b03c0 con 0x558009aed2f0 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b1830 con 0x558009af54d0 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.822+0000 7f91eb39f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b0a90 con 0x558009ad4d80 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91eb39f740 10 client.0 _ll_register_callbacks cb 0x7f91cc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009f33a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009f33a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:49020/0 (socket says 172.21.15.112:49020) 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 -- 172.21.15.112:0/1896930117 learned_addr learned my addr 172.21.15.112:0/1896930117 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e29b0640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e21af640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009ad4d80 0x558009f33510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 -- 172.21.15.112:0/1896930117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009ad4d80 msgr2=0x558009f33510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009ad4d80 0x558009f33510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 -- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 msgr2=0x558009ec7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 -- 172.21.15.112:0/1896930117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558009ec80b0 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91e19ae640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009f33a50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0a7400 tx=0x7f91cc0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f65c05d850c082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91c5ffb640 1 -- 172.21.15.112:0/1896930117 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0c6060 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91c5ffb640 1 -- 172.21.15.112:0/1896930117 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f91cc0c8380 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d402b040 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91c5ffb640 1 -- 172.21.15.112:0/1896930117 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f91cc087050 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91eb39f740 10 client.15579 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x558009f3abc0 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.823+0000 7f91c5ffb640 1 -- 172.21.15.112:0/1896930117 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d402b040 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91c5ffb640 1 -- 172.21.15.112:0/1896930117 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f91cc0a2de0 con 0x558009aed2f0 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91eb39f740 10 client.15579 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91eb39f740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91eb39f740 20 client.15579 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009b161b0 0x558009f3d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91eb39f740 4 client.15579 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558009e23ee0 con 0x558009b161b0 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91e21af640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009b161b0 0x558009f3d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91c5ffb640 10 client.15579 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.824+0000 7f91e21af640 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009b161b0 0x558009f3d380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d600729a16c95a84 server_cookie=f29254fbffcd4fcc in_seq=0 out_seq=0 2024-04-25T16:40:42.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 125306 14.42 MB/sec warmup 48 sec latency 44.406 ms 2024-04-25T16:40:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91c5ffb640 1 -- 172.21.15.112:0/1896930117 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558009e23ee0 con 0x558009b161b0 2024-04-25T16:40:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91c5ffb640 10 client.15579 handle_command_reply: tid=0 2024-04-25T16:40:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 client.15579 shutdown 2024-04-25T16:40:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 msgr2=0x558009f33a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009f33a50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0a7400 tx=0x7f91cc0043a0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 shutdown_connections 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558009af54d0 0x558009ec7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558009aed2f0 0x558009f33a50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558009b161b0 0x558009f3d380 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.956+0000 7f91eb39f740 1 --2- 172.21.15.112:0/1896930117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558009ad4d80 0x558009f33510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91c5ffb640 0 client.15579 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 >> 172.21.15.112:0/1896930117 conn(0x558009b0a300 msgr2=0x558009eb5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 shutdown_connections 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91eb39f740 1 -- 172.21.15.112:0/1896930117 wait complete. 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91eb39f740 20 client.15579 trim_cache size 0 max 16384 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:40:42.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:40:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:40:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:40:42.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f91c80afcd0 msgr2=0x7f91c80b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f91c80afcd0 0x7f91c80b21e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f91d4020000 tx=0x7f91d4021000 comp rx=0 tx=0).stop 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 msgr2=0x7f91dc1a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc1a2ac0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00e9f0 tx=0x7f91cc00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 shutdown_connections 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f91c80afcd0 0x7f91c80b21e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91dc108f00 0x7f91dc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f91dc108a60 0x7f91dc1a2ac0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 --2- 172.21.15.112:0/3061663233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91dc1031c0 0x7f91dc1a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.957+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 >> 172.21.15.112:0/3061663233 conn(0x7f91dc0781b0 msgr2=0x7f91dc104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.958+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 shutdown_connections 2024-04-25T16:40:42.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:42.958+0000 7f91e4413640 1 -- 172.21.15.112:0/3061663233 wait complete. 2024-04-25T16:40:43.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:43 smithi112 ceph-mon[29122]: pgmap v70: 97 pgs: 97 active+clean; 51 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-04-25T16:40:43.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:43 smithi133 ceph-mon[32691]: pgmap v70: 97 pgs: 97 active+clean; 51 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-04-25T16:40:43.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:43 smithi116 ceph-mon[32712]: pgmap v70: 97 pgs: 97 active+clean; 51 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-04-25T16:40:43.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 128091 14.48 MB/sec warmup 49 sec latency 66.046 ms 2024-04-25T16:40:44.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 130938 14.45 MB/sec warmup 50 sec latency 13.171 ms 2024-04-25T16:40:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:45 smithi112 ceph-mon[29122]: pgmap v71: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 561 op/s 2024-04-25T16:40:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:45 smithi133 ceph-mon[32691]: pgmap v71: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 561 op/s 2024-04-25T16:40:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:45 smithi116 ceph-mon[32712]: pgmap v71: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 561 op/s 2024-04-25T16:40:45.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 133102 14.39 MB/sec warmup 51 sec latency 16.438 ms 2024-04-25T16:40:46.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 135254 14.38 MB/sec warmup 52 sec latency 146.178 ms 2024-04-25T16:40:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:47 smithi112 ceph-mon[29122]: pgmap v72: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-04-25T16:40:47.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:47 smithi133 ceph-mon[32691]: pgmap v72: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-04-25T16:40:47.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:47 smithi116 ceph-mon[32712]: pgmap v72: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-04-25T16:40:47.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 138179 14.38 MB/sec warmup 53 sec latency 20.703 ms 2024-04-25T16:40:48.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 140193 14.30 MB/sec warmup 54 sec latency 14.658 ms 2024-04-25T16:40:49.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:49 smithi112 ceph-mon[29122]: pgmap v73: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-04-25T16:40:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:49 smithi133 ceph-mon[32691]: pgmap v73: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-04-25T16:40:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:49 smithi116 ceph-mon[32712]: pgmap v73: 97 pgs: 97 active+clean; 39 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 341 op/s 2024-04-25T16:40:49.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 142592 14.29 MB/sec warmup 55 sec latency 17.691 ms 2024-04-25T16:40:50.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 145671 14.37 MB/sec warmup 56 sec latency 11.355 ms 2024-04-25T16:40:51.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:51 smithi112 ceph-mon[29122]: pgmap v74: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-04-25T16:40:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:51 smithi133 ceph-mon[32691]: pgmap v74: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-04-25T16:40:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:51 smithi116 ceph-mon[32712]: pgmap v74: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.9 MiB/s wr, 457 op/s 2024-04-25T16:40:51.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 147820 14.24 MB/sec warmup 57 sec latency 68.078 ms 2024-04-25T16:40:52.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 149801 14.22 MB/sec warmup 58 sec latency 19.424 ms 2024-04-25T16:40:53.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:53 smithi112 ceph-mon[29122]: pgmap v75: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 343 op/s 2024-04-25T16:40:53.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:53 smithi133 ceph-mon[32691]: pgmap v75: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 343 op/s 2024-04-25T16:40:53.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:53 smithi116 ceph-mon[32712]: pgmap v75: 97 pgs: 97 active+clean; 44 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 343 op/s 2024-04-25T16:40:53.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 152823 14.29 MB/sec warmup 59 sec latency 20.538 ms 2024-04-25T16:40:54.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 155447 14.25 MB/sec warmup 60 sec latency 13.784 ms 2024-04-25T16:40:55.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:55 smithi112 ceph-mon[29122]: pgmap v76: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 461 op/s 2024-04-25T16:40:55.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:55 smithi133 ceph-mon[32691]: pgmap v76: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 461 op/s 2024-04-25T16:40:55.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:55 smithi116 ceph-mon[32712]: pgmap v76: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 461 op/s 2024-04-25T16:40:55.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 157562 14.22 MB/sec warmup 61 sec latency 19.848 ms 2024-04-25T16:40:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 -- 172.21.15.112:0/1419681148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a4108f00 msgr2=0x7f41a4102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 --2- 172.21.15.112:0/1419681148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a4108f00 0x7f41a4102b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f418c009900 tx=0x7f418c02a410 comp rx=0 tx=0).stop 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 -- 172.21.15.112:0/1419681148 shutdown_connections 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 --2- 172.21.15.112:0/1419681148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41a41031c0 0x7f41a410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 --2- 172.21.15.112:0/1419681148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a4108f00 0x7f41a4102b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 --2- 172.21.15.112:0/1419681148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a4102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 -- 172.21.15.112:0/1419681148 >> 172.21.15.112:0/1419681148 conn(0x7f41a40781b0 msgr2=0x7f41a40785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 -- 172.21.15.112:0/1419681148 shutdown_connections 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.059+0000 7f41a957a640 1 -- 172.21.15.112:0/1419681148 wait complete. 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 Processor -- start 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 -- start start 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a41031c0 0x7f41a419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41a4108f00 0x7f41a419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4112c80 con 0x7f41a41031c0 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4112b00 con 0x7f41a4108a60 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a957a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4112e00 con 0x7f41a4108f00 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a41031c0 0x7f41a419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f419bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f419bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a419acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:51728/0 (socket says 172.21.15.112:51728) 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f419bfff640 1 -- 172.21.15.112:0/1066711592 learned_addr learned my addr 172.21.15.112:0/1066711592 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.060+0000 7f41a37fe640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41a4108f00 0x7f41a419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a2ffd640 1 -- 172.21.15.112:0/1066711592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41a4108f00 msgr2=0x7f41a419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a2ffd640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41a4108f00 0x7f41a419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a2ffd640 1 -- 172.21.15.112:0/1066711592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 msgr2=0x7f41a419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a2ffd640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a2ffd640 1 -- 172.21.15.112:0/1066711592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a419f5d0 con 0x7f41a41031c0 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f419bfff640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a2ffd640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a41031c0 0x7f41a419a770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f41a4103750 tx=0x7f419400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a0ff9640 1 -- 172.21.15.112:0/1066711592 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419400cc40 con 0x7f41a41031c0 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a0ff9640 1 -- 172.21.15.112:0/1066711592 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419400cde0 con 0x7f41a41031c0 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a419f8c0 con 0x7f41a41031c0 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a0ff9640 1 -- 172.21.15.112:0/1066711592 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194018780 con 0x7f41a41031c0 2024-04-25T16:40:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.061+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a41a7240 con 0x7f41a41031c0 2024-04-25T16:40:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.064+0000 7f41a0ff9640 1 -- 172.21.15.112:0/1066711592 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f4194010a90 con 0x7f41a41031c0 2024-04-25T16:40:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.064+0000 7f41b042a740 1 Processor -- start 2024-04-25T16:40:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.064+0000 7f41b042a740 1 -- start start 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41b042a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41b042a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2aa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41b042a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2aa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41b042a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d5f210460 con 0x555d5ef0a1b0 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41b042a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555d5f1ca590 con 0x555d5eee94d0 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41b042a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555d5f20c420 con 0x555d5eee12f0 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2a9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37226/0 (socket says 172.21.15.112:37226) 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f419bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2aa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 -- 172.21.15.112:0/1300493153 learned_addr learned my addr 172.21.15.112:0/1300493153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a2ffd640 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 -- 172.21.15.112:0/1300493153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 msgr2=0x555d5f2aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2aa030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 -- 172.21.15.112:0/1300493153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 msgr2=0x555d5f2aa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2aa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a0ff9640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f41900afc50 0x7f41900b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 -- 172.21.15.112:0/1300493153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5f1d70d0 con 0x555d5ef0a1b0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f419bfff640 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a2ffd640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f41900afc50 0x7f41900b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a37fe640 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2a9af0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f419c009fd0 tx=0x7f419c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d235ea785a94d872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.065+0000 7f41a0ff9640 1 -- 172.21.15.112:0/1066711592 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f41940df320 con 0x7f41a41031c0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.066+0000 7f41997fa640 1 -- 172.21.15.112:0/1300493153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c004d20 con 0x555d5ef0a1b0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.066+0000 7f41997fa640 1 -- 172.21.15.112:0/1300493153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c004050 con 0x555d5ef0a1b0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.066+0000 7f41997fa640 1 -- 172.21.15.112:0/1300493153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c005280 con 0x555d5ef0a1b0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 -- 172.21.15.112:0/1300493153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 msgr2=0x555d5f2a9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2a9af0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f419c009fd0 tx=0x7f419c0118d0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41a2ffd640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f41900afc50 0x7f41900b2160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41940dcb00 tx=0x7f41940e2e60 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 -- 172.21.15.112:0/1300493153 shutdown_connections 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 --2- 172.21.15.112:0/1300493153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2a9af0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 -- 172.21.15.112:0/1300493153 >> 172.21.15.112:0/1300493153 conn(0x555d5eefe300 msgr2=0x555d5f2a62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 -- 172.21.15.112:0/1300493153 shutdown_connections 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 -- 172.21.15.112:0/1300493153 wait complete. 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 -- start start 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.067+0000 7f41b042a740 1 Processor -- start 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x555d5f2b6c20 con 0x555d5ef0a1b0 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555d5f197b60 con 0x555d5eee94d0 2024-04-25T16:40:56.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x555d5f2b6da0 con 0x555d5eee12f0 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41b042a740 10 client.0 _ll_register_callbacks cb 0x555d5f1bc2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f419bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:36924/0 (socket says 172.21.15.112:36924) 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 -- 172.21.15.112:0/2859840566 learned_addr learned my addr 172.21.15.112:0/2859840566 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f419bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2b46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:51750/0 (socket says 172.21.15.112:51750) 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 -- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 msgr2=0x555d5f2b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 -- 172.21.15.112:0/2859840566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 msgr2=0x555d5f2ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2ba110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 -- 172.21.15.112:0/2859840566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5ef22210 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f419bfff640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f41a37fe640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2b41b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f419c01b040 tx=0x7f419c009be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f71582d76e2fe860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f418a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.068+0000 7f418a7fc640 1 -- 172.21.15.112:0/2859840566 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0047d0 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419c004050 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f418a7fc640 1 -- 172.21.15.112:0/2859840566 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f419c004a40 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f418a7fc640 1 -- 172.21.15.112:0/2859840566 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f419c036050 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 10 client.25243 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555d5efee620 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f418a7fc640 1 -- 172.21.15.112:0/2859840566 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c004050 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f418a7fc640 1 -- 172.21.15.112:0/2859840566 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f419c00fce0 con 0x555d5eee12f0 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 10 client.25243 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 10 client.25243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 20 client.25243 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:56.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5eec8d80 0x555d5f2c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 4 client.25243 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d5f21e070 con 0x555d5eec8d80 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.069+0000 7f419bfff640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5eec8d80 0x555d5f2c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.070+0000 7f418a7fc640 10 client.25243 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.070+0000 7f419bfff640 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5eec8d80 0x555d5f2c6670 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7e6a6f59033627 server_cookie=3a8364cba690464a in_seq=0 out_seq=0 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.071+0000 7f418a7fc640 1 -- 172.21.15.112:0/2859840566 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555d5f21e070 con 0x555d5eec8d80 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.071+0000 7f418a7fc640 10 client.25243 handle_command_reply: tid=0 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.071+0000 7f41b042a740 1 client.25243 shutdown 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 msgr2=0x555d5f2b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2b41b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f419c01b040 tx=0x7f419c009be0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 shutdown_connections 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5ef0a1b0 0x555d5f2ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5eec8d80 0x555d5f2c6670 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 --2- 172.21.15.112:0/2859840566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee12f0 0x555d5f2b41b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f418a7fc640 0 client.25243 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 >> 172.21.15.112:0/2859840566 conn(0x555d5eefe300 msgr2=0x555d5f2a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 shutdown_connections 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 1 -- 172.21.15.112:0/2859840566 wait complete. 2024-04-25T16:40:56.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.072+0000 7f41b042a740 20 client.25243 trim_cache size 0 max 16384 2024-04-25T16:40:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 Processor -- start 2024-04-25T16:40:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 -- start start 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee94d0 0x555d5f2ba010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5ef0a1b0 0x555d5f2cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41b2420 con 0x555d5eee12f0 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41afcf0 con 0x555d5ef0a1b0 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.087+0000 7f41b042a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41adb40 con 0x555d5eee94d0 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2a9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37240/0 (socket says 172.21.15.112:37240) 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 -- 172.21.15.112:0/986558345 learned_addr learned my addr 172.21.15.112:0/986558345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f419bfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee94d0 0x555d5f2ba010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5ef0a1b0 0x555d5f2cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 -- 172.21.15.112:0/986558345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee94d0 msgr2=0x555d5f2ba010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee94d0 0x555d5f2ba010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 -- 172.21.15.112:0/986558345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5ef0a1b0 msgr2=0x555d5f2cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5ef0a1b0 0x555d5f2cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 -- 172.21.15.112:0/986558345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d5f2b4eb0 con 0x555d5eee12f0 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f419bfff640 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee94d0 0x555d5f2ba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f41a37fe640 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2a9f90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f419c000f80 tx=0x7f419c005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d865e992b41fff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f4198ff9640 1 -- 172.21.15.112:0/986558345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c004080 con 0x555d5eee12f0 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f4198ff9640 1 -- 172.21.15.112:0/986558345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c02d070 con 0x555d5eee12f0 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.088+0000 7f4198ff9640 1 -- 172.21.15.112:0/986558345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c004cd0 con 0x555d5eee12f0 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 -- 172.21.15.112:0/986558345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 msgr2=0x555d5f2a9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2a9f90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f419c000f80 tx=0x7f419c005cf0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 -- 172.21.15.112:0/986558345 shutdown_connections 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5eee94d0 0x555d5f2ba010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5ef0a1b0 0x555d5f2cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 --2- 172.21.15.112:0/986558345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2a9f90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 -- 172.21.15.112:0/986558345 >> 172.21.15.112:0/986558345 conn(0x555d5eefe300 msgr2=0x555d5f2a9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 -- 172.21.15.112:0/986558345 shutdown_connections 2024-04-25T16:40:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.089+0000 7f41b042a740 1 -- 172.21.15.112:0/986558345 wait complete. 2024-04-25T16:40:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 -- start start 2024-04-25T16:40:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 Processor -- start 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2c0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2baf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5ef0a1b0 0x555d5f2bb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41ae1b0 con 0x555d5eee12f0 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41af620 con 0x555d5eee94d0 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41ae880 con 0x555d5ef0a1b0 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41b042a740 10 client.0 _ll_register_callbacks cb 0x7f419c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2c0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37242/0 (socket says 172.21.15.112:37242) 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 -- 172.21.15.112:0/465288008 learned_addr learned my addr 172.21.15.112:0/465288008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:40:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f419bfff640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5ef0a1b0 0x555d5f2bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 -- 172.21.15.112:0/465288008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5ef0a1b0 msgr2=0x555d5f2bb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5ef0a1b0 0x555d5f2bb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 -- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 msgr2=0x555d5f2baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 -- 172.21.15.112:0/465288008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d5f2bbaf0 con 0x555d5eee12f0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f41a37fe640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2c0960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f419c004590 tx=0x7f419c003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2332823b05688bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.090+0000 7f418a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f418a7fc640 1 -- 172.21.15.112:0/465288008 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c004b50 con 0x555d5eee12f0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f418a7fc640 1 -- 172.21.15.112:0/465288008 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f419c004cf0 con 0x555d5eee12f0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419c02d070 con 0x555d5eee12f0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f418a7fc640 1 -- 172.21.15.112:0/465288008 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f419c03e070 con 0x555d5eee12f0 2024-04-25T16:40:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 10 client.15597 fetch_fsmap learned FSMap version 22 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x555d5f32d650 con 0x555d5eee12f0 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f418a7fc640 1 -- 172.21.15.112:0/465288008 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f419c02d070 con 0x555d5eee12f0 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f418a7fc640 1 -- 172.21.15.112:0/465288008 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f419c00f8f0 con 0x555d5eee12f0 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 10 client.15597 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 20 client.15597 populate_metadata read hostname 'smithi112' 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5ee9e1c0 0x555d5f32fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 4 client.15597 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.091+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d5f217ee0 con 0x555d5ee9e1c0 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.092+0000 7f419bfff640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5ee9e1c0 0x555d5f32fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.092+0000 7f419bfff640 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5ee9e1c0 0x555d5f32fca0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60c2e281b273b9c server_cookie=423142d4d771596b in_seq=0 out_seq=0 2024-04-25T16:40:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.092+0000 7f418a7fc640 10 client.15597 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f418a7fc640 1 -- 172.21.15.112:0/465288008 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d5f217ee0 con 0x555d5ee9e1c0 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f418a7fc640 10 client.15597 handle_command_reply: tid=0 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 client.15597 shutdown 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 msgr2=0x555d5f2c0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2c0960 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f419c004590 tx=0x7f419c003d70 comp rx=0 tx=0).stop 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 shutdown_connections 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x555d5ee9e1c0 0x555d5f32fca0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555d5ef0a1b0 0x555d5f2bb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555d5eee94d0 0x555d5f2baf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f418a7fc640 0 client.15597 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:40:56.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 --2- 172.21.15.112:0/465288008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x555d5eee12f0 0x555d5f2c0960 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 >> 172.21.15.112:0/465288008 conn(0x555d5eefe300 msgr2=0x555d5f2b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 shutdown_connections 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 1 -- 172.21.15.112:0/465288008 wait complete. 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.225+0000 7f41b042a740 20 client.15597 trim_cache size 0 max 16384 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:40:56.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:40:56.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f41900afc50 msgr2=0x7f41900b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f41900afc50 0x7f41900b2160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41940dcb00 tx=0x7f41940e2e60 comp rx=0 tx=0).stop 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a41031c0 msgr2=0x7f41a419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a41031c0 0x7f41a419a770 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f41a4103750 tx=0x7f419400ee50 comp rx=0 tx=0).stop 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 shutdown_connections 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f41900afc50 0x7f41900b2160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41a4108f00 0x7f41a419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41a4108a60 0x7f41a419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 --2- 172.21.15.112:0/1066711592 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41a41031c0 0x7f41a419a770 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 >> 172.21.15.112:0/1066711592 conn(0x7f41a40781b0 msgr2=0x7f41a406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 shutdown_connections 2024-04-25T16:40:56.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:40:56.226+0000 7f41a957a640 1 -- 172.21.15.112:0/1066711592 wait complete. 2024-04-25T16:40:56.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 159866 14.21 MB/sec warmup 62 sec latency 93.083 ms 2024-04-25T16:40:57.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:57 smithi112 ceph-mon[29122]: pgmap v77: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 233 op/s 2024-04-25T16:40:57.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:57 smithi133 ceph-mon[32691]: pgmap v77: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 233 op/s 2024-04-25T16:40:57.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:57 smithi116 ceph-mon[32712]: pgmap v77: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 233 op/s 2024-04-25T16:40:57.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 162131 14.11 MB/sec warmup 63 sec latency 29.595 ms 2024-04-25T16:40:58.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 164319 14.10 MB/sec warmup 64 sec latency 21.110 ms 2024-04-25T16:40:59.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:40:59 smithi112 ceph-mon[29122]: pgmap v78: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 233 op/s 2024-04-25T16:40:59.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:40:59 smithi133 ceph-mon[32691]: pgmap v78: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 233 op/s 2024-04-25T16:40:59.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:40:59 smithi116 ceph-mon[32712]: pgmap v78: 97 pgs: 97 active+clean; 49 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 233 op/s 2024-04-25T16:40:59.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 167126 14.15 MB/sec warmup 65 sec latency 11.884 ms 2024-04-25T16:41:00.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 168982 14.02 MB/sec warmup 66 sec latency 28.890 ms 2024-04-25T16:41:01.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:01 smithi112 ceph-mon[29122]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 463 op/s 2024-04-25T16:41:01.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:01 smithi133 ceph-mon[32691]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 463 op/s 2024-04-25T16:41:01.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:01 smithi116 ceph-mon[32712]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.9 MiB/s wr, 463 op/s 2024-04-25T16:41:01.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 170875 14.01 MB/sec warmup 67 sec latency 84.031 ms 2024-04-25T16:41:02.870 INFO:tasks.workunit.client.0.smithi112.stdout: 1 173827 14.06 MB/sec warmup 68 sec latency 38.980 ms 2024-04-25T16:41:03.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:03 smithi112 ceph-mon[29122]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-04-25T16:41:03.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:03 smithi133 ceph-mon[32691]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-04-25T16:41:03.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:03 smithi116 ceph-mon[32712]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 347 op/s 2024-04-25T16:41:03.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 175648 13.96 MB/sec warmup 69 sec latency 56.186 ms 2024-04-25T16:41:04.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 178032 13.96 MB/sec warmup 70 sec latency 14.523 ms 2024-04-25T16:41:05.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:05 smithi112 ceph-mon[29122]: pgmap v81: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-04-25T16:41:05.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:05 smithi133 ceph-mon[32691]: pgmap v81: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-04-25T16:41:05.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:05 smithi116 ceph-mon[32712]: pgmap v81: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 483 op/s 2024-04-25T16:41:05.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 181114 14.03 MB/sec warmup 71 sec latency 14.792 ms 2024-04-25T16:41:06.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 183361 13.94 MB/sec warmup 72 sec latency 23.034 ms 2024-04-25T16:41:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:07 smithi112 ceph-mon[29122]: pgmap v82: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-04-25T16:41:07.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:07 smithi133 ceph-mon[32691]: pgmap v82: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-04-25T16:41:07.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:07 smithi116 ceph-mon[32712]: pgmap v82: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-04-25T16:41:07.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 185144 13.91 MB/sec warmup 73 sec latency 68.348 ms 2024-04-25T16:41:08.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 188160 13.96 MB/sec warmup 74 sec latency 19.185 ms 2024-04-25T16:41:09.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 -- 172.21.15.112:0/3865058442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c107980 msgr2=0x7f518c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 --2- 172.21.15.112:0/3865058442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c107980 0x7f518c102db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f517c009900 tx=0x7f517c02a410 comp rx=0 tx=0).stop 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 -- 172.21.15.112:0/3865058442 shutdown_connections 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 --2- 172.21.15.112:0/3865058442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c10d470 0x7f518c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 --2- 172.21.15.112:0/3865058442 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518c1032f0 0x7f518c103700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 --2- 172.21.15.112:0/3865058442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c107980 0x7f518c102db0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 -- 172.21.15.112:0/3865058442 >> 172.21.15.112:0/3865058442 conn(0x7f518c0778b0 msgr2=0x7f518c077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 -- 172.21.15.112:0/3865058442 shutdown_connections 2024-04-25T16:41:09.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.330+0000 7f51913b7640 1 -- 172.21.15.112:0/3865058442 wait complete. 2024-04-25T16:41:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 Processor -- start 2024-04-25T16:41:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 -- start start 2024-04-25T16:41:09.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 0x7f518c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c107980 0x7f518c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518c10d470 0x7f518c19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f518c112c50 con 0x7f518c1032f0 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f518c112ad0 con 0x7f518c10d470 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f51913b7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f518c112dd0 con 0x7f518c107980 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f518affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 0x7f518c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f518affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 0x7f518c19b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53276/0 (socket says 172.21.15.112:53276) 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f518affd640 1 -- 172.21.15.112:0/2845846551 learned_addr learned my addr 172.21.15.112:0/2845846551 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f5183fff640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c107980 0x7f518c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.331+0000 7f518b7fe640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518c10d470 0x7f518c19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5183fff640 1 -- 172.21.15.112:0/2845846551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518c10d470 msgr2=0x7f518c19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5183fff640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518c10d470 0x7f518c19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5183fff640 1 -- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 msgr2=0x7f518c19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5183fff640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 0x7f518c19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5183fff640 1 -- 172.21.15.112:0/2845846551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1a68b0 con 0x7f518c107980 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f518affd640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 0x7f518c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5183fff640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c107980 0x7f518c19bab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f517400e9c0 tx=0x7f517400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5188ff9640 1 -- 172.21.15.112:0/2845846551 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517400ccc0 con 0x7f518c107980 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1a6ba0 con 0x7f518c107980 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5188ff9640 1 -- 172.21.15.112:0/2845846551 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51740108e0 con 0x7f518c107980 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f5188ff9640 1 -- 172.21.15.112:0/2845846551 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740187a0 con 0x7f518c107980 2024-04-25T16:41:09.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.332+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1a70f0 con 0x7f518c107980 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.334+0000 7f519826e740 1 Processor -- start 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.334+0000 7f519826e740 1 -- start start 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f5188ff9640 1 -- 172.21.15.112:0/2845846551 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f5174018940 con 0x7f518c107980 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f519826e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 0x55ea97a53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f519826e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a52960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f519826e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a52ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f519826e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ea979b9460 con 0x55ea9768a2f0 2024-04-25T16:41:09.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f519826e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea97973590 con 0x55ea976b31b0 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f519826e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ea979b5420 con 0x55ea976924d0 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f518affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a52960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 0x55ea97a53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 0x55ea97a53fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:55674/0 (socket says 172.21.15.112:55674) 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f518b7fe640 1 -- 172.21.15.112:0/697613270 learned_addr learned my addr 172.21.15.112:0/697613270 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.335+0000 7f5183fff640 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a52ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518b7fe640 1 -- 172.21.15.112:0/697613270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 msgr2=0x55ea97a52ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518b7fe640 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a52ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518b7fe640 1 -- 172.21.15.112:0/697613270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 msgr2=0x55ea97a52960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518b7fe640 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a52960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518b7fe640 1 -- 172.21.15.112:0/697613270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea979800d0 con 0x55ea976b31b0 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518affd640 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a52960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f5188ff9640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51780afc50 0x7f51780b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518affd640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51780afc50 0x7f51780b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f518b7fe640 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 0x55ea97a53fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5184009f30 tx=0x7f51840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc25311e0ae5ee90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f5188ff9640 1 -- 172.21.15.112:0/2845846551 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f51740df400 con 0x7f518c107980 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f51817fa640 1 -- 172.21.15.112:0/697613270 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184004d20 con 0x55ea976b31b0 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f51817fa640 1 -- 172.21.15.112:0/697613270 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004050 con 0x55ea976b31b0 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.336+0000 7f51817fa640 1 -- 172.21.15.112:0/697613270 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184005260 con 0x55ea976b31b0 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.337+0000 7f519826e740 1 -- 172.21.15.112:0/697613270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 msgr2=0x55ea97a53fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.337+0000 7f518affd640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51780afc50 0x7f51780b2160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f517c031440 tx=0x7f517c0058d0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.337+0000 7f519826e740 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 0x55ea97a53fc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5184009f30 tx=0x7f51840118d0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.337+0000 7f519826e740 1 -- 172.21.15.112:0/697613270 shutdown_connections 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.337+0000 7f519826e740 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a52ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.337+0000 7f519826e740 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a52960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.338+0000 7f519826e740 1 --2- 172.21.15.112:0/697613270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976b31b0 0x55ea97a53fc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.338+0000 7f519826e740 1 -- 172.21.15.112:0/697613270 >> 172.21.15.112:0/697613270 conn(0x55ea976a7300 msgr2=0x55ea97a4f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.338+0000 7f519826e740 1 -- 172.21.15.112:0/697613270 shutdown_connections 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.338+0000 7f519826e740 1 -- 172.21.15.112:0/697613270 wait complete. 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 -- start start 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 Processor -- start 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea9768a2f0 0x55ea97a65d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 0x55ea97a662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976b31b0 0x55ea97a6b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ea97a68560 con 0x55ea976924d0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ea97940b60 con 0x55ea9768a2f0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ea97a686e0 con 0x55ea976b31b0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f519826e740 10 client.0 _ll_register_callbacks cb 0x55ea979652b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f518affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea9768a2f0 0x55ea97a65d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f5183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 0x55ea97a662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f518affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea9768a2f0 0x55ea97a65d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:55682/0 (socket says 172.21.15.112:55682) 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f5183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 0x55ea97a662b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53306/0 (socket says 172.21.15.112:53306) 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f518affd640 1 -- 172.21.15.112:0/2236964997 learned_addr learned my addr 172.21.15.112:0/2236964997 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.339+0000 7f518b7fe640 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976b31b0 0x55ea97a6b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f5183fff640 1 -- 172.21.15.112:0/2236964997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976b31b0 msgr2=0x55ea97a6b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f5183fff640 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976b31b0 0x55ea97a6b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f5183fff640 1 -- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea9768a2f0 msgr2=0x55ea97a65d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f5183fff640 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea9768a2f0 0x55ea97a65d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f5183fff640 1 -- 172.21.15.112:0/2236964997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea976cb210 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f5183fff640 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 0x55ea97a662b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f51740dcb30 tx=0x7f5174009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a6eeedc6859107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.340+0000 7f51727fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.341+0000 7f51727fc640 1 -- 172.21.15.112:0/2236964997 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740a6f80 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f51727fc640 1 -- 172.21.15.112:0/2236964997 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f51740a38c0 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5184004050 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f51727fc640 1 -- 172.21.15.112:0/2236964997 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f51740a6da0 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 10 client.15609 fetch_fsmap learned FSMap version 22 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ea97797620 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f51727fc640 1 -- 172.21.15.112:0/2236964997 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5184004050 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f51727fc640 1 -- 172.21.15.112:0/2236964997 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f51740a6890 con 0x55ea976924d0 2024-04-25T16:41:09.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 10 client.15609 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 20 client.15609 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea97671d80 0x55ea97a77fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 4 client.15609 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea979c7070 con 0x55ea97671d80 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.342+0000 7f518affd640 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea97671d80 0x55ea97a77fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.343+0000 7f51727fc640 10 client.15609 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:09.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.343+0000 7f518affd640 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea97671d80 0x55ea97a77fd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223ae9940a8ee731 server_cookie=1e0333c2d06a1fce in_seq=0 out_seq=0 2024-04-25T16:41:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.344+0000 7f51727fc640 1 -- 172.21.15.112:0/2236964997 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ea979c7070 con 0x55ea97671d80 2024-04-25T16:41:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.344+0000 7f51727fc640 10 client.15609 handle_command_reply: tid=0 2024-04-25T16:41:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 client.15609 shutdown 2024-04-25T16:41:09.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 msgr2=0x55ea97a662b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 0x55ea97a662b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f51740dcb30 tx=0x7f5174009700 comp rx=0 tx=0).stop 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 shutdown_connections 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea97671d80 0x55ea97a77fd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976b31b0 0x55ea97a6b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea976924d0 0x55ea97a662b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f51727fc640 0 client.15609 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 --2- 172.21.15.112:0/2236964997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea9768a2f0 0x55ea97a65d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 >> 172.21.15.112:0/2236964997 conn(0x55ea976a7300 msgr2=0x55ea97a4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 shutdown_connections 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 1 -- 172.21.15.112:0/2236964997 wait complete. 2024-04-25T16:41:09.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.345+0000 7f519826e740 20 client.15609 trim_cache size 0 max 16384 2024-04-25T16:41:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.360+0000 7f519826e740 1 Processor -- start 2024-04-25T16:41:09.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.360+0000 7f519826e740 1 -- start start 2024-04-25T16:41:09.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f519826e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea97671d80 0x55ea97a6b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f519826e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a63540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f519826e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a7d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f519826e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1b01f0 con 0x55ea9768a2f0 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f519826e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1adac0 con 0x55ea97671d80 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f519826e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1ab910 con 0x55ea976924d0 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a63540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a63540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53320/0 (socket says 172.21.15.112:53320) 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 -- 172.21.15.112:0/4263910376 learned_addr learned my addr 172.21.15.112:0/4263910376 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f518b7fe640 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a7d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f518affd640 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea97671d80 0x55ea97a6b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 -- 172.21.15.112:0/4263910376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 msgr2=0x55ea97a7d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a7d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 -- 172.21.15.112:0/4263910376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea97671d80 msgr2=0x55ea97a6b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea97671d80 0x55ea97a6b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f5183fff640 1 -- 172.21.15.112:0/4263910376 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea97a4fea0 con 0x55ea9768a2f0 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.361+0000 7f518affd640 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea97671d80 0x55ea97a6b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.362+0000 7f5183fff640 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a63540 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f51740a7330 tx=0x7f517400efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5b5da79f58d6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.362+0000 7f5180ff9640 1 -- 172.21.15.112:0/4263910376 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740a6f80 con 0x55ea9768a2f0 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.362+0000 7f5180ff9640 1 -- 172.21.15.112:0/4263910376 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51740a7140 con 0x55ea9768a2f0 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.362+0000 7f5180ff9640 1 -- 172.21.15.112:0/4263910376 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51740a3a40 con 0x55ea9768a2f0 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- 172.21.15.112:0/4263910376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 msgr2=0x55ea97a63540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a63540 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f51740a7330 tx=0x7f517400efb0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- 172.21.15.112:0/4263910376 shutdown_connections 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea976924d0 0x55ea97a7d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a63540 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- 172.21.15.112:0/4263910376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea97671d80 0x55ea97a6b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- 172.21.15.112:0/4263910376 >> 172.21.15.112:0/4263910376 conn(0x55ea976a7300 msgr2=0x55ea97a64430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:09.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- 172.21.15.112:0/4263910376 shutdown_connections 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- 172.21.15.112:0/4263910376 wait complete. 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- start start 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 Processor -- start 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 0x55ea97a6fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a70000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976924d0 0x55ea97a70540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1abf80 con 0x55ea9768a2f0 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1ad3f0 con 0x55ea976924d0 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f518c1ac650 con 0x55ea97671d80 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.363+0000 7f519826e740 10 client.0 _ll_register_callbacks cb 0x7f51740e2c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a70000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f518affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 0x55ea97a6fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a70000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53332/0 (socket says 172.21.15.112:53332) 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f518b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976924d0 0x55ea97a70540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 -- 172.21.15.112:0/384373851 learned_addr learned my addr 172.21.15.112:0/384373851 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f518affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 0x55ea97a6fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:43162/0 (socket says 172.21.15.112:43162) 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 -- 172.21.15.112:0/384373851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 msgr2=0x55ea97a6fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 0x55ea97a6fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 -- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976924d0 msgr2=0x55ea97a70540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976924d0 0x55ea97a70540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 -- 172.21.15.112:0/384373851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea97adf340 con 0x55ea9768a2f0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f518affd640 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 0x55ea97a6fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f5183fff640 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a70000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f51740a6c60 tx=0x7f5174009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e50d59740abbc0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f51727fc640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f51727fc640 1 -- 172.21.15.112:0/384373851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f517408a060 con 0x55ea9768a2f0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f51727fc640 1 -- 172.21.15.112:0/384373851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f51740c8340 con 0x55ea9768a2f0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51740a7140 con 0x55ea9768a2f0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f51727fc640 1 -- 172.21.15.112:0/384373851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f51740a6f80 con 0x55ea9768a2f0 2024-04-25T16:41:09.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f519826e740 10 client.15615 fetch_fsmap learned FSMap version 22 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.364+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x55ea97adf340 con 0x55ea9768a2f0 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f51727fc640 1 -- 172.21.15.112:0/384373851 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51740a7140 con 0x55ea9768a2f0 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f51727fc640 1 -- 172.21.15.112:0/384373851 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f51740a3110 con 0x55ea9768a2f0 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f519826e740 10 client.15615 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f519826e740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f519826e740 20 client.15615 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f519826e740 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea976b31b0 0x55ea97ae2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f519826e740 4 client.15615 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea979c0ee0 con 0x55ea976b31b0 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f518affd640 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea976b31b0 0x55ea97ae2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f51727fc640 10 client.15615 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.365+0000 7f518affd640 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea976b31b0 0x55ea97ae2850 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ec16bfe5e59b4c server_cookie=2e09e5865f50dee7 in_seq=0 out_seq=0 2024-04-25T16:41:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f51727fc640 1 -- 172.21.15.112:0/384373851 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ea979c0ee0 con 0x55ea976b31b0 2024-04-25T16:41:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f51727fc640 10 client.15615 handle_command_reply: tid=0 2024-04-25T16:41:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 client.15615 shutdown 2024-04-25T16:41:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 msgr2=0x55ea97a70000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a70000 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f51740a6c60 tx=0x7f5174009700 comp rx=0 tx=0).stop 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 shutdown_connections 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ea976924d0 0x55ea97a70540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ea9768a2f0 0x55ea97a70000 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ea976b31b0 0x55ea97ae2850 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 --2- 172.21.15.112:0/384373851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea97671d80 0x55ea97a6fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 >> 172.21.15.112:0/384373851 conn(0x55ea976a7300 msgr2=0x55ea97a67130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f51727fc640 0 client.15615 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 shutdown_connections 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.511+0000 7f519826e740 1 -- 172.21.15.112:0/384373851 wait complete. 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f519826e740 20 client.15615 trim_cache size 0 max 16384 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:41:09.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:41:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:41:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:41:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51780afc50 msgr2=0x7f51780b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51780afc50 0x7f51780b2160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f517c031440 tx=0x7f517c0058d0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c107980 msgr2=0x7f518c19bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c107980 0x7f518c19bab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f517400e9c0 tx=0x7f517400ee80 comp rx=0 tx=0).stop 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 shutdown_connections 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f51780afc50 0x7f51780b2160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f518c107980 0x7f518c19bab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518c10d470 0x7f518c19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 --2- 172.21.15.112:0/2845846551 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f518c1032f0 0x7f518c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 >> 172.21.15.112:0/2845846551 conn(0x7f518c0778b0 msgr2=0x7f518c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.512+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 shutdown_connections 2024-04-25T16:41:09.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:09.513+0000 7f51913b7640 1 -- 172.21.15.112:0/2845846551 wait complete. 2024-04-25T16:41:09.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:09 smithi112 ceph-mon[29122]: pgmap v83: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-04-25T16:41:09.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:09 smithi133 ceph-mon[32691]: pgmap v83: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-04-25T16:41:09.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:09 smithi116 ceph-mon[32712]: pgmap v83: 97 pgs: 97 active+clean; 47 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 365 op/s 2024-04-25T16:41:09.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 189781 13.86 MB/sec warmup 75 sec latency 63.081 ms 2024-04-25T16:41:10.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 192270 13.87 MB/sec warmup 76 sec latency 24.592 ms 2024-04-25T16:41:11.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:11 smithi112 ceph-mon[29122]: pgmap v84: 97 pgs: 97 active+clean; 53 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-04-25T16:41:11.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:11 smithi133 ceph-mon[32691]: pgmap v84: 97 pgs: 97 active+clean; 53 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-04-25T16:41:11.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:11 smithi116 ceph-mon[32712]: pgmap v84: 97 pgs: 97 active+clean; 53 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 476 op/s 2024-04-25T16:41:11.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 195322 13.93 MB/sec warmup 77 sec latency 8.259 ms 2024-04-25T16:41:12.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 196967 13.82 MB/sec warmup 78 sec latency 55.203 ms 2024-04-25T16:41:13.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:13 smithi112 ceph-mon[29122]: pgmap v85: 97 pgs: 97 active+clean; 53 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-04-25T16:41:13.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:13 smithi133 ceph-mon[32691]: pgmap v85: 97 pgs: 97 active+clean; 53 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-04-25T16:41:13.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:13 smithi116 ceph-mon[32712]: pgmap v85: 97 pgs: 97 active+clean; 53 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 246 op/s 2024-04-25T16:41:13.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 199371 13.82 MB/sec warmup 79 sec latency 14.215 ms 2024-04-25T16:41:14.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 202455 13.88 MB/sec warmup 80 sec latency 8.272 ms 2024-04-25T16:41:15.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:15 smithi112 ceph-mon[29122]: pgmap v86: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-04-25T16:41:15.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:15 smithi133 ceph-mon[32691]: pgmap v86: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-04-25T16:41:15.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:15 smithi116 ceph-mon[32712]: pgmap v86: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.8 MiB/s wr, 481 op/s 2024-04-25T16:41:15.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 204783 13.81 MB/sec warmup 81 sec latency 94.651 ms 2024-04-25T16:41:16.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 206788 13.80 MB/sec warmup 82 sec latency 20.552 ms 2024-04-25T16:41:17.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:17 smithi112 ceph-mon[29122]: pgmap v87: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-04-25T16:41:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:17 smithi133 ceph-mon[32691]: pgmap v87: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-04-25T16:41:17.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:17 smithi116 ceph-mon[32712]: pgmap v87: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-04-25T16:41:17.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 209434 13.83 MB/sec warmup 83 sec latency 90.341 ms 2024-04-25T16:41:18.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 211849 13.78 MB/sec warmup 84 sec latency 13.195 ms 2024-04-25T16:41:19.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:19 smithi133 ceph-mon[32691]: pgmap v88: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-04-25T16:41:19.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:19 smithi116 ceph-mon[32712]: pgmap v88: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-04-25T16:41:19.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:19 smithi112 ceph-mon[29122]: pgmap v88: 97 pgs: 97 active+clean; 46 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 346 op/s 2024-04-25T16:41:19.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 213877 13.77 MB/sec warmup 85 sec latency 21.577 ms 2024-04-25T16:41:20.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 216815 13.81 MB/sec warmup 86 sec latency 15.582 ms 2024-04-25T16:41:21.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:21 smithi133 ceph-mon[32691]: pgmap v89: 97 pgs: 97 active+clean; 52 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-04-25T16:41:21.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:21 smithi116 ceph-mon[32712]: pgmap v89: 97 pgs: 97 active+clean; 52 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-04-25T16:41:21.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:21 smithi112 ceph-mon[29122]: pgmap v89: 97 pgs: 97 active+clean; 52 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.8 MiB/s wr, 456 op/s 2024-04-25T16:41:21.871 INFO:tasks.workunit.client.0.smithi112.stdout: 1 219809 13.82 MB/sec warmup 87 sec latency 14.292 ms 2024-04-25T16:41:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.627+0000 7f123c8af640 1 -- 172.21.15.112:0/1039349767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238109bf0 msgr2=0x7f1238101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.627+0000 7f123c8af640 1 --2- 172.21.15.112:0/1039349767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238109bf0 0x7f1238101b60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1228009900 tx=0x7f122802a410 comp rx=0 tx=0).stop 2024-04-25T16:41:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 -- 172.21.15.112:0/1039349767 shutdown_connections 2024-04-25T16:41:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 --2- 172.21.15.112:0/1039349767 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12381020a0 0x7f12381024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 --2- 172.21.15.112:0/1039349767 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 0x7f12381109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 --2- 172.21.15.112:0/1039349767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1238109bf0 0x7f1238101b60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 -- 172.21.15.112:0/1039349767 >> 172.21.15.112:0/1039349767 conn(0x7f12380778b0 msgr2=0x7f1238077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 -- 172.21.15.112:0/1039349767 shutdown_connections 2024-04-25T16:41:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.628+0000 7f123c8af640 1 -- 172.21.15.112:0/1039349767 wait complete. 2024-04-25T16:41:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 Processor -- start 2024-04-25T16:41:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 -- start start 2024-04-25T16:41:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 0x7f123819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1238109bf0 0x7f123819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 0x7f123819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1238113d60 con 0x7f12381020a0 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1238113be0 con 0x7f123810e5a0 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123c8af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1238113ee0 con 0x7f1238109bf0 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 0x7f123819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 0x7f123819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53042/0 (socket says 172.21.15.112:53042) 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f123659c640 1 -- 172.21.15.112:0/645429972 learned_addr learned my addr 172.21.15.112:0/645429972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f122ffff640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1238109bf0 0x7f123819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.629+0000 7f1236d9d640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 0x7f123819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123659c640 1 -- 172.21.15.112:0/645429972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1238109bf0 msgr2=0x7f123819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123659c640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1238109bf0 0x7f123819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123659c640 1 -- 172.21.15.112:0/645429972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 msgr2=0x7f123819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123659c640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 0x7f123819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.630 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123659c640 1 -- 172.21.15.112:0/645429972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123819f600 con 0x7f12381020a0 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f1236d9d640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 0x7f123819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123659c640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 0x7f123819a7a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f122802a930 tx=0x7f1228031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f122f7fe640 1 -- 172.21.15.112:0/645429972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122803f030 con 0x7f12381020a0 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f122f7fe640 1 -- 172.21.15.112:0/645429972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1228004490 con 0x7f12381020a0 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123819f890 con 0x7f12381020a0 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f122f7fe640 1 -- 172.21.15.112:0/645429972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228035070 con 0x7f12381020a0 2024-04-25T16:41:22.631 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.630+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123819fdd0 con 0x7f12381020a0 2024-04-25T16:41:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.633+0000 7f1243765740 1 Processor -- start 2024-04-25T16:41:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.633+0000 7f1243765740 1 -- start start 2024-04-25T16:41:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.633+0000 7f122f7fe640 1 -- 172.21.15.112:0/645429972 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f1228042030 con 0x7f12381020a0 2024-04-25T16:41:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.633+0000 7f1243765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d471b0 0x5578e70e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f1243765740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d1e2f0 0x5578e70e7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f1243765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70e7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f1243765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578e704d460 con 0x5578e6d264d0 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f1243765740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5578e7007590 con 0x5578e6d1e2f0 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f1243765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578e7049420 con 0x5578e6d471b0 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f123659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d1e2f0 0x5578e70e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70e7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53044/0 (socket says 172.21.15.112:53044) 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 -- 172.21.15.112:0/126526227 learned_addr learned my addr 172.21.15.112:0/126526227 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:22.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f1236d9d640 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d471b0 0x5578e70e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 -- 172.21.15.112:0/126526227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d471b0 msgr2=0x5578e70e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d471b0 0x5578e70e6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 -- 172.21.15.112:0/126526227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d1e2f0 msgr2=0x5578e70e7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.635 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d1e2f0 0x5578e70e7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f122ffff640 1 -- 172.21.15.112:0/126526227 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e70140d0 con 0x5578e6d264d0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.634+0000 7f123659c640 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d1e2f0 0x5578e70e7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f122f7fe640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f12240afcd0 0x7f12240b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f1236d9d640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f12240afcd0 0x7f12240b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f122f7fe640 1 -- 172.21.15.112:0/645429972 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f12280ff380 con 0x7f12381020a0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f122ffff640 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70e7570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1220009c60 tx=0x7f1220011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e062223713fc1dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f122cff9640 1 -- 172.21.15.112:0/126526227 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220004230 con 0x5578e6d264d0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f122cff9640 1 -- 172.21.15.112:0/126526227 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12200043d0 con 0x5578e6d264d0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.635+0000 7f122cff9640 1 -- 172.21.15.112:0/126526227 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220019670 con 0x5578e6d264d0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 -- 172.21.15.112:0/126526227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 msgr2=0x5578e70e7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70e7570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1220009c60 tx=0x7f1220011810 comp rx=0 tx=0).stop 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1236d9d640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f12240afcd0 0x7f12240b21e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x5578e70e3c60 tx=0x7f1230009500 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 -- 172.21.15.112:0/126526227 shutdown_connections 2024-04-25T16:41:22.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70e7570 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d1e2f0 0x5578e70e7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 --2- 172.21.15.112:0/126526227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d471b0 0x5578e70e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.636+0000 7f1243765740 1 -- 172.21.15.112:0/126526227 >> 172.21.15.112:0/126526227 conn(0x5578e6d3b300 msgr2=0x5578e70e32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.637+0000 7f1243765740 1 -- 172.21.15.112:0/126526227 shutdown_connections 2024-04-25T16:41:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.637+0000 7f1243765740 1 -- 172.21.15.112:0/126526227 wait complete. 2024-04-25T16:41:22.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.637+0000 7f1243765740 1 -- start start 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.637+0000 7f1243765740 1 Processor -- start 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5578e70f3c20 con 0x5578e6d264d0 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5578e6fd4b60 con 0x5578e6d471b0 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578e70f3da0 con 0x5578e6d1e2f0 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1243765740 10 client.0 _ll_register_callbacks cb 0x5578e6ff92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f122ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f122ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53052/0 (socket says 172.21.15.112:53052) 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f122ffff640 1 -- 172.21.15.112:0/2625023004 learned_addr learned my addr 172.21.15.112:0/2625023004 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:22.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.638+0000 7f1236d9d640 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f122ffff640 1 -- 172.21.15.112:0/2625023004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 msgr2=0x5578e70f1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f122ffff640 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f122ffff640 1 -- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 msgr2=0x5578e70f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f122ffff640 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f122ffff640 1 -- 172.21.15.112:0/2625023004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e6d5f210 con 0x5578e6d264d0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f122ffff640 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f1680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f12200043a0 tx=0x7f12200052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bada457ab4a8d369 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f121dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f121dffb640 1 -- 172.21.15.112:0/2625023004 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220028020 con 0x5578e6d264d0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.639+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12200043d0 con 0x5578e6d264d0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.640+0000 7f121dffb640 1 -- 172.21.15.112:0/2625023004 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f12200035f0 con 0x5578e6d264d0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.640+0000 7f121dffb640 1 -- 172.21.15.112:0/2625023004 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1220033050 con 0x5578e6d264d0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.640+0000 7f1243765740 10 client.15627 fetch_fsmap learned FSMap version 22 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.640+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5578e6e2b620 con 0x5578e6d264d0 2024-04-25T16:41:22.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f121dffb640 1 -- 172.21.15.112:0/2625023004 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12200043d0 con 0x5578e6d264d0 2024-04-25T16:41:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f121dffb640 1 -- 172.21.15.112:0/2625023004 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f1220018070 con 0x5578e6d264d0 2024-04-25T16:41:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f1243765740 10 client.15627 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:41:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f1243765740 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:22.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f1243765740 20 client.15627 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f1243765740 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6d05d80 0x5578e7103580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f1243765740 4 client.15627 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:41:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.641+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578e705b070 con 0x5578e6d05d80 2024-04-25T16:41:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.642+0000 7f123659c640 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6d05d80 0x5578e7103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.643+0000 7f123659c640 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6d05d80 0x5578e7103580 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ecd94411a6f2ad server_cookie=f340964b5d97dffa in_seq=0 out_seq=0 2024-04-25T16:41:22.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.643+0000 7f121dffb640 10 client.15627 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.643+0000 7f121dffb640 1 -- 172.21.15.112:0/2625023004 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578e705b070 con 0x5578e6d05d80 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f121dffb640 10 client.15627 handle_command_reply: tid=0 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 client.15627 shutdown 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 msgr2=0x5578e70f1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f1680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f12200043a0 tx=0x7f12200052f0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 shutdown_connections 2024-04-25T16:41:22.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f1680 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6d05d80 0x5578e7103580 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 --2- 172.21.15.112:0/2625023004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 >> 172.21.15.112:0/2625023004 conn(0x5578e6d3b300 msgr2=0x5578e70e8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.644+0000 7f121dffb640 0 client.15627 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.645+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 shutdown_connections 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.645+0000 7f1243765740 1 -- 172.21.15.112:0/2625023004 wait complete. 2024-04-25T16:41:22.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.645+0000 7f1243765740 20 client.15627 trim_cache size 0 max 16384 2024-04-25T16:41:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 Processor -- start 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 -- start start 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 0x5578e70e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d264d0 0x5578e70f7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12381b0160 con 0x5578e6d1e2f0 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ada30 con 0x5578e6d471b0 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f1243765740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ab880 con 0x5578e6d264d0 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f123659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 0x5578e70e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f123659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 0x5578e70e8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53058/0 (socket says 172.21.15.112:53058) 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f123659c640 1 -- 172.21.15.112:0/420012059 learned_addr learned my addr 172.21.15.112:0/420012059 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.673+0000 7f122ffff640 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d264d0 0x5578e70f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f1236d9d640 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f123659c640 1 -- 172.21.15.112:0/420012059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d264d0 msgr2=0x5578e70f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f123659c640 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d264d0 0x5578e70f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f123659c640 1 -- 172.21.15.112:0/420012059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 msgr2=0x5578e70f2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f123659c640 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f123659c640 1 -- 172.21.15.112:0/420012059 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e70f2410 con 0x5578e6d1e2f0 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f123659c640 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 0x5578e70e8fd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f12280c4770 tx=0x7f1228000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3a86e7bbf739dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.674+0000 7f121ffff640 1 -- 172.21.15.112:0/420012059 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228049070 con 0x5578e6d1e2f0 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.675+0000 7f121ffff640 1 -- 172.21.15.112:0/420012059 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12280c7010 con 0x5578e6d1e2f0 2024-04-25T16:41:22.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.675+0000 7f121ffff640 1 -- 172.21.15.112:0/420012059 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12280cb8b0 con 0x5578e6d1e2f0 2024-04-25T16:41:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 -- 172.21.15.112:0/420012059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 msgr2=0x5578e70e8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 0x5578e70e8fd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f12280c4770 tx=0x7f1228000bc0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 -- 172.21.15.112:0/420012059 shutdown_connections 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d264d0 0x5578e70f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e70f2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 --2- 172.21.15.112:0/420012059 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d1e2f0 0x5578e70e8fd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 -- 172.21.15.112:0/420012059 >> 172.21.15.112:0/420012059 conn(0x5578e6d3b300 msgr2=0x5578e70f2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 -- 172.21.15.112:0/420012059 shutdown_connections 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 -- 172.21.15.112:0/420012059 wait complete. 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.676+0000 7f1243765740 1 -- start start 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 Processor -- start 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e7172030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f123659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12381abef0 con 0x5578e6d264d0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f123659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:42890/0 (socket says 172.21.15.112:42890) 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f123659c640 1 -- 172.21.15.112:0/1310912258 learned_addr learned my addr 172.21.15.112:0/1310912258 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f122ffff640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ad360 con 0x5578e6d471b0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12381ac5c0 con 0x5578e6d1e2f0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.677+0000 7f1236d9d640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e7172030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f1243765740 10 client.0 _ll_register_callbacks cb 0x7f1228102da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f123659c640 1 -- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 msgr2=0x5578e7172030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f123659c640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e7172030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f123659c640 1 -- 172.21.15.112:0/1310912258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 msgr2=0x5578e70f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f123659c640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f123659c640 1 -- 172.21.15.112:0/1310912258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e71725a0 con 0x5578e6d1e2f0 2024-04-25T16:41:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f123659c640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f8870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12280fcb60 tx=0x7f1228003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de6af2bd053dfb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f121dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f121dffb640 1 -- 172.21.15.112:0/1310912258 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12280e1060 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f121dffb640 1 -- 172.21.15.112:0/1310912258 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7f12280c3790 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12280c7010 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f121dffb640 1 -- 172.21.15.112:0/1310912258 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1228049040 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f1243765740 10 client.25282 fetch_fsmap learned FSMap version 22 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.678+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5578e71737b0 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f121dffb640 1 -- 172.21.15.112:0/1310912258 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12280c7010 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f121dffb640 1 -- 172.21.15.112:0/1310912258 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7f12280c7760 con 0x5578e6d1e2f0 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f1243765740 10 client.25282 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f1243765740 10 client.25282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f1243765740 20 client.25282 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f1243765740 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6cdb1c0 0x5578e7175e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f1243765740 4 client.25282 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:41:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.679+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578e7054ee0 con 0x5578e6cdb1c0 2024-04-25T16:41:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.680+0000 7f122ffff640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6cdb1c0 0x5578e7175e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.681+0000 7f122ffff640 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6cdb1c0 0x5578e7175e90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6f36438ceca3f7 server_cookie=3d5a07734dd7041e in_seq=0 out_seq=0 2024-04-25T16:41:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.681+0000 7f121dffb640 10 client.25282 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:22.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:22 smithi133 ceph-mon[32691]: pgmap v90: 97 pgs: 97 active+clean; 52 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-04-25T16:41:22.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:22 smithi116 ceph-mon[32712]: pgmap v90: 97 pgs: 97 active+clean; 52 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f121dffb640 1 -- 172.21.15.112:0/1310912258 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5578e7054ee0 con 0x5578e6cdb1c0 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f121dffb640 10 client.25282 handle_command_reply: tid=0 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 client.25282 shutdown 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 msgr2=0x5578e70f8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f8870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12280fcb60 tx=0x7f1228003f60 comp rx=0 tx=0).stop 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 shutdown_connections 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5578e6d471b0 0x5578e7172030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5578e6d264d0 0x5578e70f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5578e6cdb1c0 0x5578e7175e90 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 --2- 172.21.15.112:0/1310912258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6d1e2f0 0x5578e70f8870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f121dffb640 0 client.25282 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 >> 172.21.15.112:0/1310912258 conn(0x5578e6d3b300 msgr2=0x5578e70f3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 shutdown_connections 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 1 -- 172.21.15.112:0/1310912258 wait complete. 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.817+0000 7f1243765740 20 client.25282 trim_cache size 0 max 16384 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:41:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f12240afcd0 msgr2=0x7f12240b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f12240afcd0 0x7f12240b21e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x5578e70e3c60 tx=0x7f1230009500 comp rx=0 tx=0).stop 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 msgr2=0x7f123819a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 0x7f123819a7a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f122802a930 tx=0x7f1228031a20 comp rx=0 tx=0).stop 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 shutdown_connections 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f12240afcd0 0x7f12240b21e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1238109bf0 0x7f123819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f123810e5a0 0x7f123819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 --2- 172.21.15.112:0/645429972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12381020a0 0x7f123819a7a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 >> 172.21.15.112:0/645429972 conn(0x7f12380778b0 msgr2=0x7f123810a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 shutdown_connections 2024-04-25T16:41:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:22.818+0000 7f123c8af640 1 -- 172.21.15.112:0/645429972 wait complete. 2024-04-25T16:41:22.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:22 smithi112 ceph-mon[29122]: pgmap v90: 97 pgs: 97 active+clean; 52 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-04-25T16:41:22.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 220939 13.72 MB/sec warmup 88 sec latency 94.215 ms 2024-04-25T16:41:23.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 223893 13.77 MB/sec warmup 89 sec latency 9.222 ms 2024-04-25T16:41:24.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 226875 13.78 MB/sec warmup 90 sec latency 13.146 ms 2024-04-25T16:41:24.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:24 smithi133 ceph-mon[32691]: pgmap v91: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 563 op/s 2024-04-25T16:41:25.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:24 smithi116 ceph-mon[32712]: pgmap v91: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 563 op/s 2024-04-25T16:41:25.098 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:24 smithi112 ceph-mon[29122]: pgmap v91: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.8 MiB/s wr, 563 op/s 2024-04-25T16:41:25.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 229196 13.74 MB/sec warmup 91 sec latency 22.939 ms 2024-04-25T16:41:26.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 231349 13.74 MB/sec warmup 92 sec latency 18.096 ms 2024-04-25T16:41:26.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:26 smithi133 ceph-mon[32691]: pgmap v92: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:41:27.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:26 smithi116 ceph-mon[32712]: pgmap v92: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:41:27.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:26 smithi112 ceph-mon[29122]: pgmap v92: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:41:27.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 234157 13.78 MB/sec warmup 93 sec latency 80.717 ms 2024-04-25T16:41:28.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 236571 13.73 MB/sec warmup 94 sec latency 13.069 ms 2024-04-25T16:41:28.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:28 smithi133 ceph-mon[32691]: pgmap v93: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:41:29.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:28 smithi116 ceph-mon[32712]: pgmap v93: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:41:29.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:28 smithi112 ceph-mon[29122]: pgmap v93: 97 pgs: 97 active+clean; 41 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:41:29.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 238605 13.71 MB/sec warmup 95 sec latency 18.625 ms 2024-04-25T16:41:30.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 241548 13.76 MB/sec warmup 96 sec latency 10.912 ms 2024-04-25T16:41:30.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:30 smithi133 ceph-mon[32691]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-04-25T16:41:31.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:30 smithi116 ceph-mon[32712]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-04-25T16:41:31.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:30 smithi112 ceph-mon[29122]: pgmap v94: 97 pgs: 97 active+clean; 48 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.7 MiB/s wr, 442 op/s 2024-04-25T16:41:31.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 244218 13.75 MB/sec warmup 97 sec latency 18.329 ms 2024-04-25T16:41:32.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 245755 13.69 MB/sec warmup 98 sec latency 56.989 ms 2024-04-25T16:41:32.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:32 smithi133 ceph-mon[32691]: pgmap v95: 97 pgs: 97 active+clean; 48 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-04-25T16:41:33.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:32 smithi116 ceph-mon[32712]: pgmap v95: 97 pgs: 97 active+clean; 48 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-04-25T16:41:33.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:32 smithi112 ceph-mon[29122]: pgmap v95: 97 pgs: 97 active+clean; 48 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-04-25T16:41:33.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 248633 13.73 MB/sec warmup 99 sec latency 14.541 ms 2024-04-25T16:41:34.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 251248 13.72 MB/sec warmup 100 sec latency 13.883 ms 2024-04-25T16:41:34.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:34 smithi133 ceph-mon[32691]: pgmap v96: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 429 op/s 2024-04-25T16:41:35.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:34 smithi116 ceph-mon[32712]: pgmap v96: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 429 op/s 2024-04-25T16:41:35.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:34 smithi112 ceph-mon[29122]: pgmap v96: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 429 op/s 2024-04-25T16:41:35.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 253030 13.67 MB/sec warmup 101 sec latency 26.573 ms 2024-04-25T16:41:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 -- 172.21.15.112:0/3661135470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe81031c0 msgr2=0x7ffbe810f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/3661135470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe81031c0 0x7ffbe810f4f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0009ca0 tx=0x7ffbe002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 -- 172.21.15.112:0/3661135470 shutdown_connections 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/3661135470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108f00 0x7ffbe8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/3661135470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe81031c0 0x7ffbe810f4f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/3661135470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbe8108a60 0x7ffbe8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 -- 172.21.15.112:0/3661135470 >> 172.21.15.112:0/3661135470 conn(0x7ffbe80781b0 msgr2=0x7ffbe80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 -- 172.21.15.112:0/3661135470 shutdown_connections 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.917+0000 7ffbee4a7640 1 -- 172.21.15.112:0/3661135470 wait complete. 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.918+0000 7ffbee4a7640 1 Processor -- start 2024-04-25T16:41:35.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.918+0000 7ffbee4a7640 1 -- start start 2024-04-25T16:41:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.918+0000 7ffbee4a7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbe81031c0 0x7ffbe819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.918+0000 7ffbee4a7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108a60 0x7ffbe819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbee4a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 0x7ffbe81a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbee4a7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112ca0 con 0x7ffbe8108f00 2024-04-25T16:41:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbee4a7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112b20 con 0x7ffbe81031c0 2024-04-25T16:41:35.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbee4a7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8112e20 con 0x7ffbe8108a60 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 0x7ffbe81a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbe81031c0 0x7ffbe819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 0x7ffbe81a6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33526/0 (socket says 172.21.15.112:33526) 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 -- 172.21.15.112:0/782053061 learned_addr learned my addr 172.21.15.112:0/782053061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108a60 0x7ffbe819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 -- 172.21.15.112:0/782053061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108a60 msgr2=0x7ffbe819bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108a60 0x7ffbe819bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 -- 172.21.15.112:0/782053061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbe81031c0 msgr2=0x7ffbe819b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbe81031c0 0x7ffbe819b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 -- 172.21.15.112:0/782053061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe819c360 con 0x7ffbe8108f00 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108a60 0x7ffbe819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbeca44640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 0x7ffbe81a6620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005cc0 tx=0x7ffbe0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbe57fa640 1 -- 172.21.15.112:0/782053061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0003b70 con 0x7ffbe8108f00 2024-04-25T16:41:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbe57fa640 1 -- 172.21.15.112:0/782053061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbe00318f0 con 0x7ffbe8108f00 2024-04-25T16:41:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe81a6bc0 con 0x7ffbe8108f00 2024-04-25T16:41:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbe57fa640 1 -- 172.21.15.112:0/782053061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe002f950 con 0x7ffbe8108f00 2024-04-25T16:41:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.919+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe81a7110 con 0x7ffbe8108f00 2024-04-25T16:41:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.922+0000 7ffbf5430740 1 Processor -- start 2024-04-25T16:41:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.922+0000 7ffbe57fa640 1 -- 172.21.15.112:0/782053061 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7ffbe002fb50 con 0x7ffbe8108f00 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 -- start start 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x56425289aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 0x56425289b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x56425289b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564252801460 con 0x5642524da4d0 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5642527bb590 con 0x5642524fb1b0 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbf5430740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5642527fd420 con 0x5642524d22f0 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x56425289b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 0x56425289b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 0x56425289b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:51150/0 (socket says 172.21.15.112:51150) 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 -- 172.21.15.112:0/4187435812 learned_addr learned my addr 172.21.15.112:0/4187435812 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x56425289aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 -- 172.21.15.112:0/4187435812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 msgr2=0x56425289aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x56425289aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe57fa640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ffbd00afc50 0x7ffbd00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 -- 172.21.15.112:0/4187435812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 msgr2=0x56425289b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x56425289b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 -- 172.21.15.112:0/4187435812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642527c80d0 con 0x5642524d22f0 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x56425289b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbeca44640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ffbd00afc50 0x7ffbd00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.923+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 0x56425289b030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c60 tx=0x7ffbd8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1485468fa1a22aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbe57fa640 1 -- 172.21.15.112:0/782053061 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7ffbe0100340 con 0x7ffbe8108f00 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbce7fc640 1 -- 172.21.15.112:0/4187435812 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8004230 con 0x5642524d22f0 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbce7fc640 1 -- 172.21.15.112:0/4187435812 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8009e10 con 0x5642524d22f0 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbce7fc640 1 -- 172.21.15.112:0/4187435812 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8019800 con 0x5642524d22f0 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbf5430740 1 -- 172.21.15.112:0/4187435812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 msgr2=0x56425289b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbeca44640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ffbd00afc50 0x7ffbd00b2160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00c5840 tx=0x7ffbe0095e30 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.924+0000 7ffbf5430740 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 0x56425289b030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009c60 tx=0x7ffbd8011810 comp rx=0 tx=0).stop 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 -- 172.21.15.112:0/4187435812 shutdown_connections 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524d22f0 0x56425289b030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x56425289b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 --2- 172.21.15.112:0/4187435812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x56425289aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 -- 172.21.15.112:0/4187435812 >> 172.21.15.112:0/4187435812 conn(0x5642524ef300 msgr2=0x5642528972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 -- 172.21.15.112:0/4187435812 shutdown_connections 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 -- 172.21.15.112:0/4187435812 wait complete. 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 -- start start 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.925+0000 7ffbf5430740 1 Processor -- start 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524d22f0 0x5642528a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x5642528a5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 0x5642528ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5642528a7c20 con 0x5642524da4d0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564252788b60 con 0x5642524d22f0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5642528a7da0 con 0x5642524fb1b0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 10 client.0 _ll_register_callbacks cb 0x5642527ad2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 0x5642528ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x5642528a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 0x5642528ab0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:51154/0 (socket says 172.21.15.112:51154) 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 -- 172.21.15.112:0/2832114483 learned_addr learned my addr 172.21.15.112:0/2832114483 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524d22f0 0x5642528a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 -- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524d22f0 msgr2=0x5642528a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524d22f0 0x5642528a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 -- 172.21.15.112:0/2832114483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 msgr2=0x5642528a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x5642528a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.930 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 -- 172.21.15.112:0/2832114483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564252513210 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 0x5642528ab0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011fa0 tx=0x7ffbdc009af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c97214d39bab84b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbc77fe640 1 -- 172.21.15.112:0/2832114483 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc01b040 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbc77fe640 1 -- 172.21.15.112:0/2832114483 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7ffbdc004050 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd8009e10 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.926+0000 7ffbc77fe640 1 -- 172.21.15.112:0/2832114483 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc004250 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 10 client.25294 fetch_fsmap learned FSMap version 22 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x5642525df620 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbc77fe640 1 -- 172.21.15.112:0/2832114483 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8009e10 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbc77fe640 1 -- 172.21.15.112:0/2832114483 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ffbdc02b480 con 0x5642524fb1b0 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 10 client.25294 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 10 client.25294 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 20 client.25294 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5642524b9d80 0x5642528b7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 4 client.25294 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:41:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425280f070 con 0x5642524b9d80 2024-04-25T16:41:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5642524b9d80 0x5642528b7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5642524b9d80 0x5642528b7700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e67b19011d647bd7 server_cookie=f0f5a6e05b6ae7b2 in_seq=0 out_seq=0 2024-04-25T16:41:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.927+0000 7ffbc77fe640 10 client.25294 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.929+0000 7ffbc77fe640 1 -- 172.21.15.112:0/2832114483 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56425280f070 con 0x5642524b9d80 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.929+0000 7ffbc77fe640 10 client.25294 handle_command_reply: tid=0 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.929+0000 7ffbf5430740 1 client.25294 shutdown 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.929+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 msgr2=0x5642528ab0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.929+0000 7ffbf5430740 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 0x5642528ab0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011fa0 tx=0x7ffbdc009af0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 shutdown_connections 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5642524b9d80 0x5642528b7700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524fb1b0 0x5642528ab0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524da4d0 0x5642528a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbc77fe640 0 client.25294 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 --2- 172.21.15.112:0/2832114483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524d22f0 0x5642528a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 >> 172.21.15.112:0/2832114483 conn(0x5642524ef300 msgr2=0x56425289a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 shutdown_connections 2024-04-25T16:41:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 1 -- 172.21.15.112:0/2832114483 wait complete. 2024-04-25T16:41:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.930+0000 7ffbf5430740 20 client.25294 trim_cache size 0 max 16384 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.947+0000 7ffbf5430740 1 Processor -- start 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.947+0000 7ffbf5430740 1 -- start start 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbf5430740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81b0250 con 0x5642524d22f0 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbf5430740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81adb20 con 0x5642524fb1b0 2024-04-25T16:41:35.947 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbf5430740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ab970 con 0x5642524da4d0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528a6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:51164/0 (socket says 172.21.15.112:51164) 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 -- 172.21.15.112:0/1286150465 learned_addr learned my addr 172.21.15.112:0/1286150465 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 -- 172.21.15.112:0/1286150465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 msgr2=0x5642528bceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528bceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 -- 172.21.15.112:0/1286150465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 msgr2=0x5642528ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe7fff640 1 -- 172.21.15.112:0/1286150465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425289b8a0 con 0x5642524da4d0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.948+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528bceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.949+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528a6d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8019470 tx=0x7ffbd80193e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22195b43d5089249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.949+0000 7ffbcdffb640 1 -- 172.21.15.112:0/1286150465 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd801f040 con 0x5642524da4d0 2024-04-25T16:41:35.948 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.949+0000 7ffbcdffb640 1 -- 172.21.15.112:0/1286150465 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8034070 con 0x5642524da4d0 2024-04-25T16:41:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.949+0000 7ffbcdffb640 1 -- 172.21.15.112:0/1286150465 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8024600 con 0x5642524da4d0 2024-04-25T16:41:35.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:35 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:41:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 -- 172.21.15.112:0/1286150465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 msgr2=0x5642528a6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528a6d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8019470 tx=0x7ffbd80193e0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 -- 172.21.15.112:0/1286150465 shutdown_connections 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528a6d80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528bceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 --2- 172.21.15.112:0/1286150465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 -- 172.21.15.112:0/1286150465 >> 172.21.15.112:0/1286150465 conn(0x5642524ef300 msgr2=0x5642528a6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 -- 172.21.15.112:0/1286150465 shutdown_connections 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 -- 172.21.15.112:0/1286150465 wait complete. 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 -- start start 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.950+0000 7ffbf5430740 1 Processor -- start 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81abfe0 con 0x5642524d22f0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ad450 con 0x5642524fb1b0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ac6b0 con 0x5642524da4d0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbf5430740 10 client.0 _ll_register_callbacks cb 0x7ffbdc009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33554/0 (socket says 172.21.15.112:33554) 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 -- 172.21.15.112:0/447491679 learned_addr learned my addr 172.21.15.112:0/447491679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 -- 172.21.15.112:0/447491679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 msgr2=0x5642528aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 -- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 msgr2=0x5642528af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 -- 172.21.15.112:0/447491679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642528afa30 con 0x5642524d22f0 2024-04-25T16:41:35.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbe77fe640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528b19f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011fd0 tx=0x7ffbdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5bc60092db3610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.951+0000 7ffbc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbc77fe640 1 -- 172.21.15.112:0/447491679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc02bc70 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbc77fe640 1 -- 172.21.15.112:0/447491679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8556+0+0 (secure 0 0 0) 0x7ffbdc005040 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd8034070 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbc77fe640 1 -- 172.21.15.112:0/447491679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc02be10 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 10 client.15651 fetch_fsmap learned FSMap version 22 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) -- 0x56425291fa30 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbc77fe640 1 -- 172.21.15.112:0/447491679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbd8034070 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbc77fe640 1 -- 172.21.15.112:0/447491679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 22) ==== 8338+0+0 (secure 0 0 0) 0x7ffbdc02c400 con 0x5642524d22f0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 10 client.15651 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 20 client.15651 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56425248f1c0 0x5642529221c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 4 client.15651 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564252808ee0 con 0x56425248f1c0 2024-04-25T16:41:35.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.952+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56425248f1c0 0x5642529221c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.953+0000 7ffbc77fe640 10 client.15651 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:35.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:35.953+0000 7ffbe7fff640 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56425248f1c0 0x5642529221c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49a01ece384507c server_cookie=10d902ba6b208c58 in_seq=0 out_seq=0 2024-04-25T16:41:35.996 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:35 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:41:36.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:35 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:41:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbc77fe640 1 -- 172.21.15.112:0/447491679 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564252808ee0 con 0x56425248f1c0 2024-04-25T16:41:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbc77fe640 10 client.15651 handle_command_reply: tid=0 2024-04-25T16:41:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 client.15651 shutdown 2024-04-25T16:41:36.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 msgr2=0x5642528b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528b19f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc011fd0 tx=0x7ffbdc000bc0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 shutdown_connections 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56425248f1c0 0x5642529221c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5642524da4d0 0x5642528aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbc77fe640 0 client.15651 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5642524fb1b0 0x5642528af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 --2- 172.21.15.112:0/447491679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5642524d22f0 0x5642528b19f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.108+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 >> 172.21.15.112:0/447491679 conn(0x5642524ef300 msgr2=0x5642528a7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 shutdown_connections 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbf5430740 1 -- 172.21.15.112:0/447491679 wait complete. 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbf5430740 20 client.15651 trim_cache size 0 max 16384 2024-04-25T16:41:36.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:41:36.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ffbd00afc50 msgr2=0x7ffbd00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ffbd00afc50 0x7ffbd00b2160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00c5840 tx=0x7ffbe0095e30 comp rx=0 tx=0).stop 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 msgr2=0x7ffbe81a6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.109+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 0x7ffbe81a6620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005cc0 tx=0x7ffbe0031700 comp rx=0 tx=0).stop 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 shutdown_connections 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ffbd00afc50 0x7ffbd00b2160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe8108a60 0x7ffbe819bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffbe8108f00 0x7ffbe81a6620 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 --2- 172.21.15.112:0/782053061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbe81031c0 0x7ffbe819b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 >> 172.21.15.112:0/782053061 conn(0x7ffbe80781b0 msgr2=0x7ffbe8100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 shutdown_connections 2024-04-25T16:41:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:36.110+0000 7ffbee4a7640 1 -- 172.21.15.112:0/782053061 wait complete. 2024-04-25T16:41:36.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 255656 13.70 MB/sec warmup 102 sec latency 86.395 ms 2024-04-25T16:41:36.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:36 smithi133 ceph-mon[32691]: pgmap v97: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 211 op/s 2024-04-25T16:41:36.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:36 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:41:36.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:36 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:41:36.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:36 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-mon[32712]: pgmap v97: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 211 op/s 2024-04-25T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:41:37.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:41:37.008 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setuser ceph since I am not root 2024-04-25T16:41:37.009 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setgroup ceph since I am not root 2024-04-25T16:41:37.009 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-mds[62762]: -- 172.21.15.116:0/279595248 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d794899a40 con 0x55d7948e1180 2024-04-25T16:41:37.009 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:41:36 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: starting mds.e at 2024-04-25T16:41:37.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:36 smithi112 ceph-mon[29122]: pgmap v97: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.0 MiB/s wr, 211 op/s 2024-04-25T16:41:37.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:36 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:41:37.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:36 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:41:37.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:36 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:41:37.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 257512 13.63 MB/sec warmup 103 sec latency 77.287 ms 2024-04-25T16:41:37.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:37 smithi133 ceph-mon[32691]: daemon mds.e restarted 2024-04-25T16:41:37.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:37 smithi133 ceph-mon[32691]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T16:41:37.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:37 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6834/2394788700,v1:172.21.15.116:6835/2394788700] up:boot 2024-04-25T16:41:37.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:37 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:41:37.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:41:38.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:37 smithi116 ceph-mon[32712]: daemon mds.e restarted 2024-04-25T16:41:38.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:37 smithi116 ceph-mon[32712]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T16:41:38.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:37 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6834/2394788700,v1:172.21.15.116:6835/2394788700] up:boot 2024-04-25T16:41:38.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:37 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:41:38.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:41:38.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:37 smithi112 ceph-mon[29122]: daemon mds.e restarted 2024-04-25T16:41:38.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:37 smithi112 ceph-mon[29122]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T16:41:38.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:37 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6834/2394788700,v1:172.21.15.116:6835/2394788700] up:boot 2024-04-25T16:41:38.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:37 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:41:38.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:41:38.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 259648 13.63 MB/sec warmup 104 sec latency 25.722 ms 2024-04-25T16:41:39.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:39 smithi112 ceph-mon[29122]: pgmap v99: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-04-25T16:41:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:39 smithi133 ceph-mon[32691]: pgmap v99: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-04-25T16:41:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:39 smithi116 ceph-mon[32712]: pgmap v99: 97 pgs: 97 active+clean; 63 MiB data, 734 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 253 op/s 2024-04-25T16:41:39.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 262767 13.67 MB/sec warmup 105 sec latency 8.590 ms 2024-04-25T16:41:40.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 265405 13.66 MB/sec warmup 106 sec latency 14.291 ms 2024-04-25T16:41:41.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:41 smithi112 ceph-mon[29122]: pgmap v100: 97 pgs: 97 active+clean; 55 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T16:41:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:41 smithi133 ceph-mon[32691]: pgmap v100: 97 pgs: 97 active+clean; 55 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T16:41:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:41 smithi116 ceph-mon[32712]: pgmap v100: 97 pgs: 97 active+clean; 55 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T16:41:41.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 267311 13.62 MB/sec warmup 107 sec latency 31.776 ms 2024-04-25T16:41:42.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 269974 13.65 MB/sec warmup 108 sec latency 63.741 ms 2024-04-25T16:41:43.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:43 smithi112 ceph-mon[29122]: pgmap v101: 97 pgs: 97 active+clean; 55 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T16:41:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:43 smithi133 ceph-mon[32691]: pgmap v101: 97 pgs: 97 active+clean; 55 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T16:41:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:43 smithi116 ceph-mon[32712]: pgmap v101: 97 pgs: 97 active+clean; 55 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-04-25T16:41:43.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 272602 13.64 MB/sec warmup 109 sec latency 19.905 ms 2024-04-25T16:41:44.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 274760 13.62 MB/sec warmup 110 sec latency 13.676 ms 2024-04-25T16:41:45.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:45 smithi112 ceph-mon[29122]: pgmap v102: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-04-25T16:41:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:45 smithi133 ceph-mon[32691]: pgmap v102: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-04-25T16:41:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:45 smithi116 ceph-mon[32712]: pgmap v102: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-04-25T16:41:45.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 277355 13.63 MB/sec warmup 111 sec latency 12.604 ms 2024-04-25T16:41:46.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 280346 13.66 MB/sec warmup 112 sec latency 88.718 ms 2024-04-25T16:41:47.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:47 smithi112 ceph-mon[29122]: pgmap v103: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-04-25T16:41:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:47 smithi133 ceph-mon[32691]: pgmap v103: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-04-25T16:41:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:47 smithi116 ceph-mon[32712]: pgmap v103: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.7 MiB/s wr, 406 op/s 2024-04-25T16:41:47.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 282611 13.62 MB/sec warmup 113 sec latency 40.862 ms 2024-04-25T16:41:48.872 INFO:tasks.workunit.client.0.smithi112.stdout: 1 284466 13.60 MB/sec warmup 114 sec latency 69.976 ms 2024-04-25T16:41:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/3359728056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 msgr2=0x7fb7f8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/3359728056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f8106e10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec02a410 comp rx=0 tx=0).stop 2024-04-25T16:41:49.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/3359728056 shutdown_connections 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/3359728056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f81074a0 0x7fb7f810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/3359728056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f8106e10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/3359728056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f8102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/3359728056 >> 172.21.15.112:0/3359728056 conn(0x7fb7f80778b0 msgr2=0x7fb7f8077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/3359728056 shutdown_connections 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.212+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/3359728056 wait complete. 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 Processor -- start 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 -- start start 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f819acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f81074a0 0x7fb7f819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.214 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8111b30 con 0x7fb7f81069e0 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81119b0 con 0x7fb7f8102450 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fe4a1640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8111cb0 con 0x7fb7f81074a0 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f819acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:55690/0 (socket says 172.21.15.112:55690) 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f819acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41184/0 (socket says 172.21.15.112:41184) 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2618107595 learned_addr learned my addr 172.21.15.112:0/2618107595 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.213+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f81074a0 0x7fb7f819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2618107595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f81074a0 msgr2=0x7fb7f819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f81074a0 0x7fb7f819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 msgr2=0x7fb7f819acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f819acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2618107595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f819f5c0 con 0x7fb7f8102450 2024-04-25T16:41:49.215 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f819a760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e9f0 tx=0x7fb7e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f57fa640 1 -- 172.21.15.112:0/2618107595 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800cd80 con 0x7fb7f8102450 2024-04-25T16:41:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f819f8b0 con 0x7fb7f8102450 2024-04-25T16:41:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f57fa640 1 -- 172.21.15.112:0/2618107595 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80108e0 con 0x7fb7f8102450 2024-04-25T16:41:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7f57fa640 1 -- 172.21.15.112:0/2618107595 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8018980 con 0x7fb7f8102450 2024-04-25T16:41:49.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.214+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a7230 con 0x7fb7f8102450 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7f57fa640 1 -- 172.21.15.112:0/2618107595 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fb7e8010430 con 0x7fb7f8102450 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 Processor -- start 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 -- start start 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167417aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x56167417b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x56167417b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5616740e1460 con 0x561673ddb1b0 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56167409b590 con 0x561673db22f0 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb805432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5616740dd420 con 0x561673dba4d0 2024-04-25T16:41:49.223 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7f57fa640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb7e00afc50 0x7fb7e00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7fca3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167417aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x56167417b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x56167417b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7fca3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167417aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41186/0 (socket says 172.21.15.112:41186) 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x56167417b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:56446/0 (socket says 172.21.15.112:56446) 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7fca3e640 1 -- 172.21.15.112:0/3619718277 learned_addr learned my addr 172.21.15.112:0/3619718277 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.217+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb7e00afc50 0x7fb7e00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f57fa640 1 -- 172.21.15.112:0/2618107595 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb7e80df420 con 0x7fb7f8102450 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f7fff640 1 -- 172.21.15.112:0/3619718277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 msgr2=0x56167417b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x56167417b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f7fff640 1 -- 172.21.15.112:0/3619718277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 msgr2=0x56167417aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167417aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f7fff640 1 -- 172.21.15.112:0/3619718277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616740a80d0 con 0x561673db22f0 2024-04-25T16:41:49.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x56167417b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb7e00afc50 0x7fb7e00b2160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000afd0 tx=0x7fb7f000a610 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x56167417b030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80dcb50 tx=0x7fb7e800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c0a018c99d2031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7de7fc640 1 -- 172.21.15.112:0/3619718277 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80a7ca0 con 0x561673db22f0 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7de7fc640 1 -- 172.21.15.112:0/3619718277 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80a75a0 con 0x561673db22f0 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.218+0000 7fb7de7fc640 1 -- 172.21.15.112:0/3619718277 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8018d80 con 0x561673db22f0 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 -- 172.21.15.112:0/3619718277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 msgr2=0x56167417b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x56167417b030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80dcb50 tx=0x7fb7e800cf20 comp rx=0 tx=0).stop 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 -- 172.21.15.112:0/3619718277 shutdown_connections 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x56167417b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x56167417b030 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 --2- 172.21.15.112:0/3619718277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167417aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 -- 172.21.15.112:0/3619718277 >> 172.21.15.112:0/3619718277 conn(0x561673dcf300 msgr2=0x5616741772a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 -- 172.21.15.112:0/3619718277 shutdown_connections 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 -- 172.21.15.112:0/3619718277 wait complete. 2024-04-25T16:41:49.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.219+0000 7fb805432740 1 -- start start 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 Processor -- start 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x561674186b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x561674187050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x561674184fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561674187bc0 con 0x561673dba4d0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561674068b60 con 0x561673ddb1b0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561674187d40 con 0x561673db22f0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb805432740 10 client.0 _ll_register_callbacks cb 0x56167408d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x561674187050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x561674187050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41190/0 (socket says 172.21.15.112:41190) 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 -- 172.21.15.112:0/937644806 learned_addr learned my addr 172.21.15.112:0/937644806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x561674186b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x561674184fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 -- 172.21.15.112:0/937644806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 msgr2=0x561674186b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x561674186b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 -- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 msgr2=0x561674184fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x561674184fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7f77fe640 1 -- 172.21.15.112:0/937644806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561673df3210 con 0x561673dba4d0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.220+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x561674184fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x561674187050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf03331c7a17c092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7d77fe640 1 -- 172.21.15.112:0/937644806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec035040 con 0x561673dba4d0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7d77fe640 1 -- 172.21.15.112:0/937644806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb7ec004040 con 0x561673dba4d0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e80a75a0 con 0x561673dba4d0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7d77fe640 1 -- 172.21.15.112:0/937644806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb7ec004240 con 0x561673dba4d0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 10 client.15669 fetch_fsmap learned FSMap version 23 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561673ebf620 con 0x561673dba4d0 2024-04-25T16:41:49.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7d77fe640 1 -- 172.21.15.112:0/937644806 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e80a75a0 con 0x561673dba4d0 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb7d77fe640 1 -- 172.21.15.112:0/937644806 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb7ec004ad0 con 0x561673dba4d0 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 10 client.15669 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 20 client.15669 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.221+0000 7fb805432740 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d99d80 0x5616741976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.222+0000 7fb805432740 4 client.15669 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.222+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616740ef070 con 0x561673d99d80 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.222+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d99d80 0x5616741976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.222+0000 7fb7d77fe640 10 client.15669 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.222+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d99d80 0x5616741976e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbadf5f8652aa4bf server_cookie=3630e38fae953283 in_seq=0 out_seq=0 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb7d77fe640 1 -- 172.21.15.112:0/937644806 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5616740ef070 con 0x561673d99d80 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb7d77fe640 10 client.15669 handle_command_reply: tid=0 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 client.15669 shutdown 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 msgr2=0x561674187050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x561674187050 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec031740 comp rx=0 tx=0).stop 2024-04-25T16:41:49.228 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 shutdown_connections 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x561674184fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x561674187050 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d99d80 0x5616741976e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 --2- 172.21.15.112:0/937644806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x561674186b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb7d77fe640 0 client.15669 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 >> 172.21.15.112:0/937644806 conn(0x561673dcf300 msgr2=0x56167417a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 shutdown_connections 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 1 -- 172.21.15.112:0/937644806 wait complete. 2024-04-25T16:41:49.229 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.223+0000 7fb805432740 20 client.15669 trim_cache size 0 max 16384 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 Processor -- start 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 -- start start 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x561674183880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x561674183dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167418cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81b2420 con 0x561673ddb1b0 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81afcf0 con 0x561673db22f0 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb805432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81adb40 con 0x561673dba4d0 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167418cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x561674183880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167418cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41198/0 (socket says 172.21.15.112:41198) 2024-04-25T16:41:49.240 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 -- 172.21.15.112:0/2643377954 learned_addr learned my addr 172.21.15.112:0/2643377954 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x561674183dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 -- 172.21.15.112:0/2643377954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 msgr2=0x561674183dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x561674183dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 -- 172.21.15.112:0/2643377954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 msgr2=0x561674183880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x561674183880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 -- 172.21.15.112:0/2643377954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561674180b70 con 0x561673ddb1b0 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x561674183880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.240+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167418cfd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000a380 tx=0x7fb7f0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5035ba152dc1e1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.241+0000 7fb7ddffb640 1 -- 172.21.15.112:0/2643377954 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0003d40 con 0x561673ddb1b0 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.241+0000 7fb7ddffb640 1 -- 172.21.15.112:0/2643377954 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f0025070 con 0x561673ddb1b0 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.241+0000 7fb7ddffb640 1 -- 172.21.15.112:0/2643377954 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000e450 con 0x561673ddb1b0 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 -- 172.21.15.112:0/2643377954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 msgr2=0x56167418cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167418cfd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000a380 tx=0x7fb7f0009000 comp rx=0 tx=0).stop 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 -- 172.21.15.112:0/2643377954 shutdown_connections 2024-04-25T16:41:49.241 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673dba4d0 0x561674183dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673ddb1b0 0x56167418cfd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 --2- 172.21.15.112:0/2643377954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673db22f0 0x561674183880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 -- 172.21.15.112:0/2643377954 >> 172.21.15.112:0/2643377954 conn(0x561673dcf300 msgr2=0x56167417e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 -- 172.21.15.112:0/2643377954 shutdown_connections 2024-04-25T16:41:49.242 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 -- 172.21.15.112:0/2643377954 wait complete. 2024-04-25T16:41:49.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 -- start start 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.242+0000 7fb805432740 1 Processor -- start 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x56167418a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x56167418ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x56167418cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81ae1b0 con 0x561673dba4d0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81af620 con 0x561673ddb1b0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81ae880 con 0x561673db22f0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb805432740 10 client.0 _ll_register_callbacks cb 0x7fb7ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x56167418a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x56167418ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x56167418a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:56476/0 (socket says 172.21.15.112:56476) 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2896100226 learned_addr learned my addr 172.21.15.112:0/2896100226 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7fca3e640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x56167418cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 msgr2=0x56167418cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x56167418cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2896100226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 msgr2=0x56167418ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x56167418ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 -- 172.21.15.112:0/2896100226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167418d110 con 0x561673db22f0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x56167418ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7f7fff640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x56167418a610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800c6a0 tx=0x7fb7e80a3bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0be1f5ee106778a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.243+0000 7fb7d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb7d77fe640 1 -- 172.21.15.112:0/2896100226 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80a6a00 con 0x561673db22f0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f0025070 con 0x561673db22f0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb7d77fe640 1 -- 172.21.15.112:0/2896100226 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb7e8018b20 con 0x561673db22f0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb7d77fe640 1 -- 172.21.15.112:0/2896100226 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb7e80a75a0 con 0x561673db22f0 2024-04-25T16:41:49.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 10 client.25324 fetch_fsmap learned FSMap version 23 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5616742038d0 con 0x561673db22f0 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb7d77fe640 1 -- 172.21.15.112:0/2896100226 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f0025070 con 0x561673db22f0 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb7d77fe640 1 -- 172.21.15.112:0/2896100226 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb7e80c8380 con 0x561673db22f0 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 10 client.25324 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 10 client.25324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 20 client.25324 populate_metadata read hostname 'smithi112' 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d6f1c0 0x561674205bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 4 client.25324 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616740e8ee0 con 0x561673d6f1c0 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.244+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d6f1c0 0x561674205bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.245+0000 7fb7f77fe640 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d6f1c0 0x561674205bd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa57b21e1d85ad0 server_cookie=fdefbfa11660dfcc in_seq=0 out_seq=0 2024-04-25T16:41:49.247 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.245+0000 7fb7d77fe640 10 client.25324 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:49.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:49 smithi112 ceph-mon[29122]: pgmap v104: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-04-25T16:41:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb7d77fe640 1 -- 172.21.15.112:0/2896100226 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616740e8ee0 con 0x561673d6f1c0 2024-04-25T16:41:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb7d77fe640 10 client.25324 handle_command_reply: tid=0 2024-04-25T16:41:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 client.25324 shutdown 2024-04-25T16:41:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 msgr2=0x56167418a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x56167418a610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800c6a0 tx=0x7fb7e80a3bf0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 shutdown_connections 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561673ddb1b0 0x56167418cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561673dba4d0 0x56167418ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561673d6f1c0 0x561674205bd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 --2- 172.21.15.112:0/2896100226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561673db22f0 0x56167418a610 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb7d77fe640 0 client.25324 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 >> 172.21.15.112:0/2896100226 conn(0x561673dcf300 msgr2=0x5616741847f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 shutdown_connections 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.395+0000 7fb805432740 1 -- 172.21.15.112:0/2896100226 wait complete. 2024-04-25T16:41:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb805432740 20 client.25324 trim_cache size 0 max 16384 2024-04-25T16:41:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:41:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:41:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:41:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb7e00afc50 msgr2=0x7fb7e00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb7e00afc50 0x7fb7e00b2160 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000afd0 tx=0x7fb7f000a610 comp rx=0 tx=0).stop 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 msgr2=0x7fb7f819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f819a760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800e9f0 tx=0x7fb7e800eeb0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 shutdown_connections 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb7e00afc50 0x7fb7e00b2160 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7f81074a0 0x7fb7f819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7f81069e0 0x7fb7f819acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 --2- 172.21.15.112:0/2618107595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f8102450 0x7fb7f819a760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 >> 172.21.15.112:0/2618107595 conn(0x7fb7f80778b0 msgr2=0x7fb7f8100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 shutdown_connections 2024-04-25T16:41:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:41:49.396+0000 7fb7fe4a1640 1 -- 172.21.15.112:0/2618107595 wait complete. 2024-04-25T16:41:49.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:49 smithi133 ceph-mon[32691]: pgmap v104: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-04-25T16:41:49.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:49 smithi116 ceph-mon[32712]: pgmap v104: 97 pgs: 97 active+clean; 58 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 385 op/s 2024-04-25T16:41:49.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 287312 13.64 MB/sec warmup 115 sec latency 47.520 ms 2024-04-25T16:41:50.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 289532 13.58 MB/sec warmup 116 sec latency 17.597 ms 2024-04-25T16:41:51.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:51 smithi112 ceph-mon[29122]: pgmap v105: 97 pgs: 97 active+clean; 66 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-04-25T16:41:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:51 smithi133 ceph-mon[32691]: pgmap v105: 97 pgs: 97 active+clean; 66 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-04-25T16:41:51.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:51 smithi116 ceph-mon[32712]: pgmap v105: 97 pgs: 97 active+clean; 66 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.7 MiB/s wr, 450 op/s 2024-04-25T16:41:51.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 291685 13.59 MB/sec warmup 117 sec latency 14.540 ms 2024-04-25T16:41:52.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 294711 13.63 MB/sec warmup 118 sec latency 8.469 ms 2024-04-25T16:41:53.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:53 smithi112 ceph-mon[29122]: pgmap v106: 97 pgs: 97 active+clean; 66 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-25T16:41:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:53 smithi133 ceph-mon[32691]: pgmap v106: 97 pgs: 97 active+clean; 66 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-25T16:41:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:53 smithi116 ceph-mon[32712]: pgmap v106: 97 pgs: 97 active+clean; 66 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 221 op/s 2024-04-25T16:41:53.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 296911 13.58 MB/sec warmup 119 sec latency 47.659 ms 2024-04-25T16:41:55.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:55 smithi112 ceph-mon[29122]: pgmap v107: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 441 op/s 2024-04-25T16:41:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:55 smithi133 ceph-mon[32691]: pgmap v107: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 441 op/s 2024-04-25T16:41:55.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:55 smithi116 ceph-mon[32712]: pgmap v107: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 441 op/s 2024-04-25T16:41:55.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 301883 17.17 MB/sec execute 1 sec latency 15.446 ms 2024-04-25T16:41:56.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 304450 14.85 MB/sec execute 2 sec latency 14.088 ms 2024-04-25T16:41:57.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:57 smithi112 ceph-mon[29122]: pgmap v108: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-04-25T16:41:57.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:57 smithi133 ceph-mon[32691]: pgmap v108: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-04-25T16:41:57.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:57 smithi116 ceph-mon[32712]: pgmap v108: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-04-25T16:41:57.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 306475 13.79 MB/sec execute 3 sec latency 22.413 ms 2024-04-25T16:41:58.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 308845 13.78 MB/sec execute 4 sec latency 132.207 ms 2024-04-25T16:41:59.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:41:59 smithi112 ceph-mon[29122]: pgmap v109: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-04-25T16:41:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:41:59 smithi133 ceph-mon[32691]: pgmap v109: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-04-25T16:41:59.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:41:59 smithi116 ceph-mon[32712]: pgmap v109: 97 pgs: 97 active+clean; 57 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 331 op/s 2024-04-25T16:41:59.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 311215 12.69 MB/sec execute 5 sec latency 12.452 ms 2024-04-25T16:42:00.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 312590 12.42 MB/sec execute 6 sec latency 24.324 ms 2024-04-25T16:42:01.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:01 smithi112 ceph-mon[29122]: pgmap v110: 97 pgs: 97 active+clean; 65 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-04-25T16:42:01.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:01 smithi133 ceph-mon[32691]: pgmap v110: 97 pgs: 97 active+clean; 65 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-04-25T16:42:01.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:01 smithi116 ceph-mon[32712]: pgmap v110: 97 pgs: 97 active+clean; 65 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 443 op/s 2024-04-25T16:42:01.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 315168 12.45 MB/sec execute 7 sec latency 81.377 ms 2024-04-25T16:42:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 -- 172.21.15.112:0/3703556777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708103800 msgr2=0x7fb70810f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 --2- 172.21.15.112:0/3703556777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708103800 0x7fb70810f570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb700009ca0 tx=0x7fb70002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 -- 172.21.15.112:0/3703556777 shutdown_connections 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 --2- 172.21.15.112:0/3703556777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 0x7fb708103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 --2- 172.21.15.112:0/3703556777 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708103800 0x7fb70810f570 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 --2- 172.21.15.112:0/3703556777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708108a50 0x7fb708102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 -- 172.21.15.112:0/3703556777 >> 172.21.15.112:0/3703556777 conn(0x7fb7080fd8f0 msgr2=0x7fb7080ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 -- 172.21.15.112:0/3703556777 shutdown_connections 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.496+0000 7fb70d230640 1 -- 172.21.15.112:0/3703556777 wait complete. 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.497+0000 7fb70d230640 1 Processor -- start 2024-04-25T16:42:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.497+0000 7fb70d230640 1 -- start start 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708103800 0x7fb7081041c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 0x7fb708104700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 0x7fb708104c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112c60 con 0x7fb708108a50 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112ae0 con 0x7fb708103800 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112de0 con 0x7fb708108ef0 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 0x7fb708104700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708103800 0x7fb7081041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 0x7fb708104700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45802/0 (socket says 172.21.15.112:45802) 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70759e640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 0x7fb708104c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 -- 172.21.15.112:0/4191966687 learned_addr learned my addr 172.21.15.112:0/4191966687 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 -- 172.21.15.112:0/4191966687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 msgr2=0x7fb708104c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 0x7fb708104c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 -- 172.21.15.112:0/4191966687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708103800 msgr2=0x7fb7081041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708103800 0x7fb7081041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 -- 172.21.15.112:0/4191966687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a8a90 con 0x7fb708108a50 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70759e640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 0x7fb708104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb706d9d640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708103800 0x7fb7081041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70659c640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 0x7fb708104700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e9f0 tx=0x7fb6fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb6effff640 1 -- 172.21.15.112:0/4191966687 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ccc0 con 0x7fb708108a50 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a8d80 con 0x7fb708108a50 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb6effff640 1 -- 172.21.15.112:0/4191966687 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6fc0035f0 con 0x7fb708108a50 2024-04-25T16:42:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.498+0000 7fb6effff640 1 -- 172.21.15.112:0/4191966687 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ce60 con 0x7fb708108a50 2024-04-25T16:42:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.499+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a9210 con 0x7fb708108a50 2024-04-25T16:42:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 Processor -- start 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 -- start start 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb6effff640 1 -- 172.21.15.112:0/4191966687 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fb6fc004030 con 0x7fb708108a50 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa51af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 0x560fcfa52030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6914d0 0x560fcfa52570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560fcf9b8460 con 0x560fcf6892f0 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560fcf972590 con 0x560fcf6914d0 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.501+0000 7fb7141bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560fcf9b4420 con 0x560fcf6b21b0 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 0x560fcfa52030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb70659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6914d0 0x560fcfa52570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 0x560fcfa52030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45814/0 (socket says 172.21.15.112:45814) 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 -- 172.21.15.112:0/3937020555 learned_addr learned my addr 172.21.15.112:0/3937020555 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb70759e640 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa51af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 -- 172.21.15.112:0/3937020555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 msgr2=0x560fcfa51af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa51af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 -- 172.21.15.112:0/3937020555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6914d0 msgr2=0x560fcfa52570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6914d0 0x560fcfa52570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 -- 172.21.15.112:0/3937020555 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fcf97f0d0 con 0x560fcf6892f0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb70659c640 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6914d0 0x560fcfa52570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb706d9d640 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 0x560fcfa52030 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80114f0 tx=0x7fb6f80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0c4cc692d98a50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb6effff640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb6f00afc50 0x7fb6f00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb6ed7fa640 1 -- 172.21.15.112:0/3937020555 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801e070 con 0x560fcf6892f0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb70759e640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb6f00afc50 0x7fb6f00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb6ed7fa640 1 -- 172.21.15.112:0/3937020555 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f800fa70 con 0x560fcf6892f0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb6ed7fa640 1 -- 172.21.15.112:0/3937020555 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8019ba0 con 0x560fcf6892f0 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.502+0000 7fb6effff640 1 -- 172.21.15.112:0/4191966687 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb6fc0df340 con 0x7fb708108a50 2024-04-25T16:42:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 -- 172.21.15.112:0/3937020555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 msgr2=0x560fcfa52030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb70759e640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb6f00afc50 0x7fb6f00b2160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb700033040 tx=0x7fb700032eb0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 0x560fcfa52030 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80114f0 tx=0x7fb6f80119b0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 -- 172.21.15.112:0/3937020555 shutdown_connections 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6914d0 0x560fcfa52570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6892f0 0x560fcfa52030 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3937020555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa51af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 -- 172.21.15.112:0/3937020555 >> 172.21.15.112:0/3937020555 conn(0x560fcf6a6300 msgr2=0x560fcfa4e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 -- 172.21.15.112:0/3937020555 shutdown_connections 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.503+0000 7fb7141bd740 1 -- 172.21.15.112:0/3937020555 wait complete. 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 -- start start 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 Processor -- start 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6892f0 0x560fcfa5c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa5c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560fcfa5ec20 con 0x560fcf6914d0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560fcf93fb60 con 0x560fcf6892f0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560fcfa5eda0 con 0x560fcf6b21b0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb7141bd740 10 client.0 _ll_register_callbacks cb 0x560fcf9642b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa5c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa5c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45828/0 (socket says 172.21.15.112:45828) 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 -- 172.21.15.112:0/796220426 learned_addr learned my addr 172.21.15.112:0/796220426 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb706d9d640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6892f0 0x560fcfa5c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70759e640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 -- 172.21.15.112:0/796220426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 msgr2=0x560fcfa62110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa62110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 -- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6892f0 msgr2=0x560fcfa5c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6892f0 0x560fcfa5c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb70659c640 1 -- 172.21.15.112:0/796220426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fcf6ca210 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.504+0000 7fb706d9d640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6892f0 0x560fcfa5c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb70659c640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa5c750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0a71c0 tx=0x7fb6fc0a2c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31373ba77d7eaab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb6eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb6eaff5640 1 -- 172.21.15.112:0/796220426 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0a6e10 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f800fa70 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb6eaff5640 1 -- 172.21.15.112:0/796220426 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb6fc003aa0 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb6eaff5640 1 -- 172.21.15.112:0/796220426 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb6fc0c1050 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 10 client.15687 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560fcf796620 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb6eaff5640 1 -- 172.21.15.112:0/796220426 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f800fa70 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb6eaff5640 1 -- 172.21.15.112:0/796220426 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb6fc0afe90 con 0x560fcf6914d0 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 10 client.15687 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 20 client.15687 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf670d80 0x560fcfa6e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 4 client.15687 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.505+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fcf9c6070 con 0x560fcf670d80 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.506+0000 7fb706d9d640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf670d80 0x560fcfa6e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.506+0000 7fb6eaff5640 10 client.15687 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.506+0000 7fb706d9d640 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf670d80 0x560fcfa6e7e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ee78d9fcca903 server_cookie=f53edbf30ebcbaf0 in_seq=0 out_seq=0 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb6eaff5640 1 -- 172.21.15.112:0/796220426 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560fcf9c6070 con 0x560fcf670d80 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb6eaff5640 10 client.15687 handle_command_reply: tid=0 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 client.15687 shutdown 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 msgr2=0x560fcfa5c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa5c750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc0a71c0 tx=0x7fb6fc0a2c60 comp rx=0 tx=0).stop 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 shutdown_connections 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf670d80 0x560fcfa6e7e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6b21b0 0x560fcfa62110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb6eaff5640 0 client.15687 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa5c750 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 --2- 172.21.15.112:0/796220426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf6892f0 0x560fcfa5c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 >> 172.21.15.112:0/796220426 conn(0x560fcf6a6300 msgr2=0x560fcfa51920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 shutdown_connections 2024-04-25T16:42:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 1 -- 172.21.15.112:0/796220426 wait complete. 2024-04-25T16:42:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.507+0000 7fb7141bd740 20 client.15687 trim_cache size 0 max 16384 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.522+0000 7fb7141bd740 1 Processor -- start 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.522+0000 7fb7141bd740 1 -- start start 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa73f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb7141bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081b45a0 con 0x560fcf6914d0 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb7141bd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081b1e70 con 0x560fcf670d80 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb7141bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081afcc0 con 0x560fcf6892f0 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb70759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa55ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57982/0 (socket says 172.21.15.112:57982) 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb70759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45838/0 (socket says 172.21.15.112:45838) 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb70659c640 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa73f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 -- 172.21.15.112:0/1869858915 learned_addr learned my addr 172.21.15.112:0/1869858915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 -- 172.21.15.112:0/1869858915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 msgr2=0x560fcfa73f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa73f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 -- 172.21.15.112:0/1869858915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 msgr2=0x560fcfa744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 -- 172.21.15.112:0/1869858915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fcfa5b2e0 con 0x560fcf670d80 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb70659c640 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa73f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb70759e640 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb706d9d640 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa55ec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8014040 tx=0x7fb6f80197a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511ca8981c8fddbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.523+0000 7fb6ecff9640 1 -- 172.21.15.112:0/1869858915 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8029020 con 0x560fcf670d80 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb6ecff9640 1 -- 172.21.15.112:0/1869858915 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8005560 con 0x560fcf670d80 2024-04-25T16:42:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb6ecff9640 1 -- 172.21.15.112:0/1869858915 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801e040 con 0x560fcf670d80 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 -- 172.21.15.112:0/1869858915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 msgr2=0x560fcfa55ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa55ec0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8014040 tx=0x7fb6f80197a0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 -- 172.21.15.112:0/1869858915 shutdown_connections 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa73f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 --2- 172.21.15.112:0/1869858915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa55ec0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 -- 172.21.15.112:0/1869858915 >> 172.21.15.112:0/1869858915 conn(0x560fcf6a6300 msgr2=0x560fcfa557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 -- 172.21.15.112:0/1869858915 shutdown_connections 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.524+0000 7fb7141bd740 1 -- 172.21.15.112:0/1869858915 wait complete. 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 -- start start 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 Processor -- start 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa66070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa66af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081b0330 con 0x560fcf6914d0 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081b17a0 con 0x560fcf670d80 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb7081b0a00 con 0x560fcf6892f0 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb7141bd740 10 client.0 _ll_register_callbacks cb 0x7fb6fc0045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb70759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa66af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa66070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa66070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57990/0 (socket says 172.21.15.112:57990) 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 -- 172.21.15.112:0/3228865330 learned_addr learned my addr 172.21.15.112:0/3228865330 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb70659c640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 -- 172.21.15.112:0/3228865330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 msgr2=0x560fcfa665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa665b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 -- 172.21.15.112:0/3228865330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 msgr2=0x560fcfa66af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa66af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb706d9d640 1 -- 172.21.15.112:0/3228865330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fcfad5b50 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb70659c640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.525+0000 7fb70759e640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa66af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb706d9d640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa66070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80097e0 tx=0x7fb6f8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f17ae70bb2529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb6eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb6eaff5640 1 -- 172.21.15.112:0/3228865330 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f801e030 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb6eaff5640 1 -- 172.21.15.112:0/3228865330 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb6f8011300 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6f8005560 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb6eaff5640 1 -- 172.21.15.112:0/3228865330 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb6f8029040 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 10 client.25391 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560fcfad6ca0 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb6eaff5640 1 -- 172.21.15.112:0/3228865330 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f8005560 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb6eaff5640 1 -- 172.21.15.112:0/3228865330 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb6f8019b70 con 0x560fcf670d80 2024-04-25T16:42:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 10 client.25391 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 10 client.25391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 20 client.25391 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.526+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf6b21b0 0x560fcfad9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.527+0000 7fb7141bd740 4 client.25391 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.527+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fcf9bfee0 con 0x560fcf6b21b0 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.527+0000 7fb70659c640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf6b21b0 0x560fcfad9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.527+0000 7fb70659c640 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf6b21b0 0x560fcfad9270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aadd065a7e2cc81 server_cookie=c605fba4cd8346a9 in_seq=0 out_seq=0 2024-04-25T16:42:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.527+0000 7fb6eaff5640 10 client.25391 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:02.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.699+0000 7fb6eaff5640 1 -- 172.21.15.112:0/3228865330 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fcf9bfee0 con 0x560fcf6b21b0 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.699+0000 7fb6eaff5640 10 client.25391 handle_command_reply: tid=0 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.699+0000 7fb7141bd740 1 client.25391 shutdown 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 msgr2=0x560fcfa66070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa66070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80097e0 tx=0x7fb6f8009310 comp rx=0 tx=0).stop 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 shutdown_connections 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560fcf6b21b0 0x560fcfad9270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560fcf6892f0 0x560fcfa665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb6eaff5640 0 client.25391 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560fcf6914d0 0x560fcfa66af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 --2- 172.21.15.112:0/3228865330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fcf670d80 0x560fcfa66070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 >> 172.21.15.112:0/3228865330 conn(0x560fcf6a6300 msgr2=0x560fcfa5a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 shutdown_connections 2024-04-25T16:42:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 1 -- 172.21.15.112:0/3228865330 wait complete. 2024-04-25T16:42:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.700+0000 7fb7141bd740 20 client.25391 trim_cache size 0 max 16384 2024-04-25T16:42:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:42:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:42:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:42:02.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb6f00afc50 msgr2=0x7fb6f00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb6f00afc50 0x7fb6f00b2160 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb700033040 tx=0x7fb700032eb0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 msgr2=0x7fb708104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 0x7fb708104700 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e9f0 tx=0x7fb6fc00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 shutdown_connections 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb6f00afc50 0x7fb6f00b2160 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb708108ef0 0x7fb708104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb708108a50 0x7fb708104700 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 --2- 172.21.15.112:0/4191966687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb708103800 0x7fb7081041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 >> 172.21.15.112:0/4191966687 conn(0x7fb7080fd8f0 msgr2=0x7fb7081008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:02.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 shutdown_connections 2024-04-25T16:42:02.702 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:02.701+0000 7fb70d230640 1 -- 172.21.15.112:0/4191966687 wait complete. 2024-04-25T16:42:02.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 316777 11.95 MB/sec execute 8 sec latency 86.103 ms 2024-04-25T16:42:03.309 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:03 smithi116 ceph-mon[32712]: pgmap v111: 97 pgs: 97 active+clean; 65 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-04-25T16:42:03.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:03 smithi112 ceph-mon[29122]: pgmap v111: 97 pgs: 97 active+clean; 65 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-04-25T16:42:03.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:03 smithi133 ceph-mon[32691]: pgmap v111: 97 pgs: 97 active+clean; 65 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-04-25T16:42:03.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 319461 12.50 MB/sec execute 9 sec latency 83.255 ms 2024-04-25T16:42:04.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 321725 12.05 MB/sec execute 10 sec latency 13.267 ms 2024-04-25T16:42:05.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:05 smithi112 ceph-mon[29122]: pgmap v112: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 589 op/s 2024-04-25T16:42:05.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:05 smithi133 ceph-mon[32691]: pgmap v112: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 589 op/s 2024-04-25T16:42:05.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:05 smithi116 ceph-mon[32712]: pgmap v112: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 589 op/s 2024-04-25T16:42:05.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 323863 12.17 MB/sec execute 11 sec latency 12.756 ms 2024-04-25T16:42:06.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 326716 12.58 MB/sec execute 12 sec latency 9.977 ms 2024-04-25T16:42:07.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:07 smithi112 ceph-mon[29122]: pgmap v113: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-04-25T16:42:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:07 smithi133 ceph-mon[32691]: pgmap v113: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-04-25T16:42:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:07 smithi116 ceph-mon[32712]: pgmap v113: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-04-25T16:42:07.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 329547 12.64 MB/sec execute 13 sec latency 13.602 ms 2024-04-25T16:42:08.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 331040 12.30 MB/sec execute 14 sec latency 72.573 ms 2024-04-25T16:42:09.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:09 smithi112 ceph-mon[29122]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-04-25T16:42:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:09 smithi133 ceph-mon[32691]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-04-25T16:42:09.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:09 smithi116 ceph-mon[32712]: pgmap v114: 97 pgs: 97 active+clean; 52 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 369 op/s 2024-04-25T16:42:09.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 333804 12.62 MB/sec execute 15 sec latency 55.817 ms 2024-04-25T16:42:10.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 336722 12.73 MB/sec execute 16 sec latency 13.284 ms 2024-04-25T16:42:11.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:11 smithi112 ceph-mon[29122]: pgmap v115: 97 pgs: 97 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 478 op/s 2024-04-25T16:42:11.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:11 smithi133 ceph-mon[32691]: pgmap v115: 97 pgs: 97 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 478 op/s 2024-04-25T16:42:11.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:11 smithi116 ceph-mon[32712]: pgmap v115: 97 pgs: 97 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.0 MiB/s wr, 478 op/s 2024-04-25T16:42:11.873 INFO:tasks.workunit.client.0.smithi112.stdout: 1 338646 12.57 MB/sec execute 17 sec latency 33.710 ms 2024-04-25T16:42:12.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 341120 12.64 MB/sec execute 18 sec latency 16.929 ms 2024-04-25T16:42:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:13 smithi112 ceph-mon[29122]: pgmap v116: 97 pgs: 97 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 365 op/s 2024-04-25T16:42:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:13 smithi133 ceph-mon[32691]: pgmap v116: 97 pgs: 97 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 365 op/s 2024-04-25T16:42:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:13 smithi116 ceph-mon[32712]: pgmap v116: 97 pgs: 97 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 365 op/s 2024-04-25T16:42:13.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 343541 12.64 MB/sec execute 19 sec latency 65.611 ms 2024-04-25T16:42:14.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 345661 12.59 MB/sec execute 20 sec latency 13.007 ms 2024-04-25T16:42:15.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:15 smithi112 ceph-mon[29122]: pgmap v117: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-04-25T16:42:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:15 smithi133 ceph-mon[32691]: pgmap v117: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-04-25T16:42:15.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:15 smithi116 ceph-mon[32712]: pgmap v117: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.8 MiB/s wr, 483 op/s 2024-04-25T16:42:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 -- 172.21.15.112:0/3215794614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108a60 msgr2=0x7fee80102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 --2- 172.21.15.112:0/3215794614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108a60 0x7fee80102610 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fee78009ca0 tx=0x7fee7802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 -- 172.21.15.112:0/3215794614 shutdown_connections 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 --2- 172.21.15.112:0/3215794614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee80108f00 0x7fee80102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 --2- 172.21.15.112:0/3215794614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee801031c0 0x7fee8010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 --2- 172.21.15.112:0/3215794614 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108a60 0x7fee80102610 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 -- 172.21.15.112:0/3215794614 >> 172.21.15.112:0/3215794614 conn(0x7fee800781b0 msgr2=0x7fee800785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 -- 172.21.15.112:0/3215794614 shutdown_connections 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.810+0000 7fee876aa640 1 -- 172.21.15.112:0/3215794614 wait complete. 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 Processor -- start 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 -- start start 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee801031c0 0x7fee8019a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee80108a60 0x7fee8019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 0x7fee8019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee80112c60 con 0x7fee80108f00 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee80112ae0 con 0x7fee80108a60 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee876aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee80112de0 con 0x7fee801031c0 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 0x7fee8019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee84c45640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee80108a60 0x7fee8019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85446640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee801031c0 0x7fee8019a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 0x7fee8019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46308/0 (socket says 172.21.15.112:46308) 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 -- 172.21.15.112:0/194224179 learned_addr learned my addr 172.21.15.112:0/194224179 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 -- 172.21.15.112:0/194224179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee801031c0 msgr2=0x7fee8019a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee801031c0 0x7fee8019a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 -- 172.21.15.112:0/194224179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee80108a60 msgr2=0x7fee8019acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee80108a60 0x7fee8019acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee85c47640 1 -- 172.21.15.112:0/194224179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8019f5d0 con 0x7fee80108f00 2024-04-25T16:42:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.811+0000 7fee84c45640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee80108a60 0x7fee8019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.812+0000 7fee85c47640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 0x7fee8019ef90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e970 tx=0x7fee7c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.812+0000 7fee767fc640 1 -- 172.21.15.112:0/194224179 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00ccc0 con 0x7fee80108f00 2024-04-25T16:42:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.812+0000 7fee767fc640 1 -- 172.21.15.112:0/194224179 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c00ce60 con 0x7fee80108f00 2024-04-25T16:42:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.812+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8019f8c0 con 0x7fee80108f00 2024-04-25T16:42:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.812+0000 7fee767fc640 1 -- 172.21.15.112:0/194224179 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c010430 con 0x7fee80108f00 2024-04-25T16:42:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.812+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8019fdc0 con 0x7fee80108f00 2024-04-25T16:42:15.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.814+0000 7fee767fc640 1 -- 172.21.15.112:0/194224179 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fee7c004ff0 con 0x7fee80108f00 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.814+0000 7fee8e634740 1 Processor -- start 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.814+0000 7fee8e634740 1 -- start start 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee8e634740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 0x556f59092af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee8e634740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 0x556f59093030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee8e634740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f59093570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee8e634740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f58ff9460 con 0x556f58cd24d0 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee8e634740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f58fb3590 con 0x556f58cf31b0 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee8e634740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556f58ff5420 con 0x556f58cca2f0 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee84c45640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 0x556f59092af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 0x556f59093030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 0x556f59093030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:38630/0 (socket says 172.21.15.112:38630) 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee84c45640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 0x556f59092af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:35874/0 (socket says 172.21.15.112:35874) 2024-04-25T16:42:15.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 -- 172.21.15.112:0/3935339794 learned_addr learned my addr 172.21.15.112:0/3935339794 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee767fc640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fee6c0afc50 0x7fee6c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee84c45640 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f59093570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85c47640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fee6c0afc50 0x7fee6c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 -- 172.21.15.112:0/3935339794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 msgr2=0x556f59092af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 0x556f59092af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 -- 172.21.15.112:0/3935339794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 msgr2=0x556f59093570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f59093570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 -- 172.21.15.112:0/3935339794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f58fc00d0 con 0x556f58cca2f0 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee84c45640 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 0x556f59092af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee767fc640 1 -- 172.21.15.112:0/194224179 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fee7c0df470 con 0x7fee80108f00 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.815+0000 7fee85446640 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 0x556f59093030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee78009ca0 tx=0x7fee78009410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1afde4b5b99a7c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:15.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee6b7fe640 1 -- 172.21.15.112:0/3935339794 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78031cf0 con 0x556f58cca2f0 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee85c47640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fee6c0afc50 0x7fee6c0b2160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0a7580 tx=0x7fee7c0ca040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee6b7fe640 1 -- 172.21.15.112:0/3935339794 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78036030 con 0x556f58cca2f0 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee6b7fe640 1 -- 172.21.15.112:0/3935339794 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78004240 con 0x556f58cca2f0 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee8e634740 1 -- 172.21.15.112:0/3935339794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 msgr2=0x556f59093030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee8e634740 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 0x556f59093030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fee78009ca0 tx=0x7fee78009410 comp rx=0 tx=0).stop 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee8e634740 1 -- 172.21.15.112:0/3935339794 shutdown_connections 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee8e634740 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cca2f0 0x556f59093030 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee8e634740 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f59093570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.816+0000 7fee8e634740 1 --2- 172.21.15.112:0/3935339794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cf31b0 0x556f59092af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- 172.21.15.112:0/3935339794 >> 172.21.15.112:0/3935339794 conn(0x556f58ce7300 msgr2=0x556f5908f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- 172.21.15.112:0/3935339794 shutdown_connections 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- 172.21.15.112:0/3935339794 wait complete. 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- start start 2024-04-25T16:42:15.823 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 Processor -- start 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f5909d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f5909d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 0x556f590a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556f5909fa40 con 0x556f58cf31b0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f58f80b60 con 0x556f58cca2f0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.817+0000 7fee8e634740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556f5909fbc0 con 0x556f58cd24d0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee8e634740 10 client.0 _ll_register_callbacks cb 0x556f58fa52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 0x556f590a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 0x556f590a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46320/0 (socket says 172.21.15.112:46320) 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 -- 172.21.15.112:0/1461678621 learned_addr learned my addr 172.21.15.112:0/1461678621 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee85446640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f5909d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f5909d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 -- 172.21.15.112:0/1461678621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 msgr2=0x556f5909d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f5909d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.824 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 -- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 msgr2=0x556f5909d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f5909d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 -- 172.21.15.112:0/1461678621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f58d0b210 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee85446640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f5909d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee84c45640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 0x556f590a2ec0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fee8006c520 tx=0x7fee70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7662691898660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee68ff9640 1 -- 172.21.15.112:0/1461678621 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70011e30 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee68ff9640 1 -- 172.21.15.112:0/1461678621 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fee7000e750 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee78036030 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee68ff9640 1 -- 172.21.15.112:0/1461678621 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fee7000e950 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee8e634740 10 client.15705 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.818+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556f58dd7620 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee68ff9640 1 -- 172.21.15.112:0/1461678621 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78036030 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee68ff9640 1 -- 172.21.15.112:0/1461678621 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fee7001d070 con 0x556f58cf31b0 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee8e634740 10 client.15705 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:15.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee8e634740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee8e634740 20 client.15705 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee8e634740 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cb1d80 0x556f590af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee8e634740 4 client.15705 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f59007070 con 0x556f58cb1d80 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee85446640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cb1d80 0x556f590af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee68ff9640 10 client.15705 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.819+0000 7fee85446640 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cb1d80 0x556f590af4f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c9726917ca79d server_cookie=6bf359603b15acb9 in_seq=0 out_seq=0 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.820+0000 7fee68ff9640 1 -- 172.21.15.112:0/1461678621 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f59007070 con 0x556f58cb1d80 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.820+0000 7fee68ff9640 10 client.15705 handle_command_reply: tid=0 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.820+0000 7fee8e634740 1 client.15705 shutdown 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 msgr2=0x556f590a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 0x556f590a2ec0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fee8006c520 tx=0x7fee70004730 comp rx=0 tx=0).stop 2024-04-25T16:42:15.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 shutdown_connections 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cb1d80 0x556f590af4f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee68ff9640 0 client.15705 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f5909d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cf31b0 0x556f590a2ec0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 --2- 172.21.15.112:0/1461678621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f5909d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 >> 172.21.15.112:0/1461678621 conn(0x556f58ce7300 msgr2=0x556f59092920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 shutdown_connections 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 1 -- 172.21.15.112:0/1461678621 wait complete. 2024-04-25T16:42:15.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.821+0000 7fee8e634740 20 client.15705 trim_cache size 0 max 16384 2024-04-25T16:42:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 Processor -- start 2024-04-25T16:42:15.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 -- start start 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cb1d80 0x556f59096e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f590b5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee801b2310 con 0x556f58cd24d0 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee801afbe0 con 0x556f58cca2f0 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee8e634740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ada30 con 0x556f58cb1d80 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee84c45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f590b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee84c45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f590b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46324/0 (socket says 172.21.15.112:46324) 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee84c45640 1 -- 172.21.15.112:0/3887468622 learned_addr learned my addr 172.21.15.112:0/3887468622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee85446640 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.836+0000 7fee84c45640 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cb1d80 0x556f59096e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee84c45640 1 -- 172.21.15.112:0/3887468622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cb1d80 msgr2=0x556f59096e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee84c45640 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cb1d80 0x556f59096e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee84c45640 1 -- 172.21.15.112:0/3887468622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 msgr2=0x556f590b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee84c45640 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590b4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee84c45640 1 -- 172.21.15.112:0/3887468622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f5909cee0 con 0x556f58cd24d0 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee84c45640 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f590b5230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fee70001eb0 tx=0x7fee700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d5506201cd2477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee6affd640 1 -- 172.21.15.112:0/3887468622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70011260 con 0x556f58cd24d0 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee6affd640 1 -- 172.21.15.112:0/3887468622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7002c480 con 0x556f58cd24d0 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.837+0000 7fee6affd640 1 -- 172.21.15.112:0/3887468622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee700238c0 con 0x556f58cd24d0 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 -- 172.21.15.112:0/3887468622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 msgr2=0x556f590b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f590b5230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fee70001eb0 tx=0x7fee700046c0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 -- 172.21.15.112:0/3887468622 shutdown_connections 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cd24d0 0x556f590b5230 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 --2- 172.21.15.112:0/3887468622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cb1d80 0x556f59096e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 -- 172.21.15.112:0/3887468622 >> 172.21.15.112:0/3887468622 conn(0x556f58ce7300 msgr2=0x556f59096740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 -- 172.21.15.112:0/3887468622 shutdown_connections 2024-04-25T16:42:15.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 -- 172.21.15.112:0/3887468622 wait complete. 2024-04-25T16:42:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 -- start start 2024-04-25T16:42:15.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.838+0000 7fee8e634740 1 Processor -- start 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 0x556f590a95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f590a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ae0a0 con 0x556f58cb1d80 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee801af510 con 0x556f58cca2f0 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ae770 con 0x556f58cd24d0 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee8e634740 10 client.0 _ll_register_callbacks cb 0x7fee700118f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee84c45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 0x556f590a95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee84c45640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 0x556f590a95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46332/0 (socket says 172.21.15.112:46332) 2024-04-25T16:42:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee84c45640 1 -- 172.21.15.112:0/725276729 learned_addr learned my addr 172.21.15.112:0/725276729 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee84c45640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f590a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 -- 172.21.15.112:0/725276729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 msgr2=0x556f590a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f590a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 -- 172.21.15.112:0/725276729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 msgr2=0x556f590a95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 0x556f590a95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 -- 172.21.15.112:0/725276729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f590a73a0 con 0x556f58cca2f0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee84c45640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 0x556f590a95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee85446640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590a6920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x556f59093ea0 tx=0x7fee7802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=741273b7fe8e5a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.839+0000 7fee68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee68ff9640 1 -- 172.21.15.112:0/725276729 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7804a040 con 0x556f58cca2f0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee68ff9640 1 -- 172.21.15.112:0/725276729 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fee78031450 con 0x556f58cca2f0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7002c480 con 0x556f58cca2f0 2024-04-25T16:42:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee68ff9640 1 -- 172.21.15.112:0/725276729 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fee78009540 con 0x556f58cca2f0 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 10 client.25409 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x556f591175b0 con 0x556f58cca2f0 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee68ff9640 1 -- 172.21.15.112:0/725276729 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7002c480 con 0x556f58cca2f0 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee68ff9640 1 -- 172.21.15.112:0/725276729 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fee78043070 con 0x556f58cca2f0 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 10 client.25409 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 10 client.25409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 20 client.25409 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cf31b0 0x556f59119ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 4 client.25409 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f59000ee0 con 0x556f58cf31b0 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.840+0000 7fee84c45640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cf31b0 0x556f59119ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.841+0000 7fee68ff9640 10 client.25409 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:15.844 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.841+0000 7fee84c45640 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cf31b0 0x556f59119ac0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=250bc122b93b24c5 server_cookie=e84fdd31854a053d in_seq=0 out_seq=0 2024-04-25T16:42:15.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 348079 12.65 MB/sec execute 21 sec latency 29.480 ms 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.986+0000 7fee68ff9640 1 -- 172.21.15.112:0/725276729 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f59000ee0 con 0x556f58cf31b0 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.986+0000 7fee68ff9640 10 client.25409 handle_command_reply: tid=0 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.986+0000 7fee8e634740 1 client.25409 shutdown 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 msgr2=0x556f590a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590a6920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x556f59093ea0 tx=0x7fee7802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 shutdown_connections 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556f58cf31b0 0x556f59119ac0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556f58cd24d0 0x556f590a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee68ff9640 0 client.25409 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:15.986 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f58cca2f0 0x556f590a6920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 --2- 172.21.15.112:0/725276729 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556f58cb1d80 0x556f590a95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 >> 172.21.15.112:0/725276729 conn(0x556f58ce7300 msgr2=0x556f59092880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 shutdown_connections 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 1 -- 172.21.15.112:0/725276729 wait complete. 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.987+0000 7fee8e634740 20 client.25409 trim_cache size 0 max 16384 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:42:15.987 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:42:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fee6c0afc50 msgr2=0x7fee6c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fee6c0afc50 0x7fee6c0b2160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0a7580 tx=0x7fee7c0ca040 comp rx=0 tx=0).stop 2024-04-25T16:42:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 msgr2=0x7fee8019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 0x7fee8019ef90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e970 tx=0x7fee7c00ed70 comp rx=0 tx=0).stop 2024-04-25T16:42:15.988 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 shutdown_connections 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fee6c0afc50 0x7fee6c0b2160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fee80108f00 0x7fee8019ef90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee80108a60 0x7fee8019acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 --2- 172.21.15.112:0/194224179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fee801031c0 0x7fee8019a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.988+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 >> 172.21.15.112:0/194224179 conn(0x7fee800781b0 msgr2=0x7fee8006a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.989+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 shutdown_connections 2024-04-25T16:42:15.989 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:15.989+0000 7fee876aa640 1 -- 172.21.15.112:0/194224179 wait complete. 2024-04-25T16:42:16.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 350538 12.65 MB/sec execute 22 sec latency 80.016 ms 2024-04-25T16:42:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:17 smithi112 ceph-mon[29122]: pgmap v118: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-25T16:42:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:17 smithi133 ceph-mon[32691]: pgmap v118: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-25T16:42:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:17 smithi116 ceph-mon[32712]: pgmap v118: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-25T16:42:17.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 352606 12.60 MB/sec execute 23 sec latency 20.371 ms 2024-04-25T16:42:18.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 354972 12.65 MB/sec execute 24 sec latency 104.553 ms 2024-04-25T16:42:19.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:19 smithi112 ceph-mon[29122]: pgmap v119: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-25T16:42:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:19 smithi133 ceph-mon[32691]: pgmap v119: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-25T16:42:19.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:19 smithi116 ceph-mon[32712]: pgmap v119: 97 pgs: 97 active+clean; 65 MiB data, 948 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 225 op/s 2024-04-25T16:42:19.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 357632 12.66 MB/sec execute 25 sec latency 13.195 ms 2024-04-25T16:42:20.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 359659 12.62 MB/sec execute 26 sec latency 14.765 ms 2024-04-25T16:42:21.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:21 smithi112 ceph-mon[29122]: pgmap v120: 97 pgs: 97 active+clean; 55 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-04-25T16:42:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:21 smithi133 ceph-mon[32691]: pgmap v120: 97 pgs: 97 active+clean; 55 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-04-25T16:42:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:21 smithi116 ceph-mon[32712]: pgmap v120: 97 pgs: 97 active+clean; 55 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 442 op/s 2024-04-25T16:42:21.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 362257 12.67 MB/sec execute 27 sec latency 13.727 ms 2024-04-25T16:42:22.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 365327 12.84 MB/sec execute 28 sec latency 12.324 ms 2024-04-25T16:42:23.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:23 smithi112 ceph-mon[29122]: pgmap v121: 97 pgs: 97 active+clean; 55 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 334 op/s 2024-04-25T16:42:23.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:23 smithi133 ceph-mon[32691]: pgmap v121: 97 pgs: 97 active+clean; 55 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 334 op/s 2024-04-25T16:42:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:23 smithi116 ceph-mon[32712]: pgmap v121: 97 pgs: 97 active+clean; 55 MiB data, 950 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.1 MiB/s wr, 334 op/s 2024-04-25T16:42:23.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 366925 12.64 MB/sec execute 29 sec latency 68.486 ms 2024-04-25T16:42:24.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 369421 12.68 MB/sec execute 30 sec latency 9.940 ms 2024-04-25T16:42:25.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:25 smithi112 ceph-mon[29122]: pgmap v122: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-04-25T16:42:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:25 smithi133 ceph-mon[32691]: pgmap v122: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-04-25T16:42:25.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:25 smithi116 ceph-mon[32712]: pgmap v122: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 439 op/s 2024-04-25T16:42:25.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 372259 12.74 MB/sec execute 31 sec latency 12.775 ms 2024-04-25T16:42:26.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 374257 12.65 MB/sec execute 32 sec latency 41.714 ms 2024-04-25T16:42:27.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:27 smithi112 ceph-mon[29122]: pgmap v123: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-04-25T16:42:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:27 smithi133 ceph-mon[32691]: pgmap v123: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-04-25T16:42:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:27 smithi116 ceph-mon[32712]: pgmap v123: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-04-25T16:42:27.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 376641 12.69 MB/sec execute 33 sec latency 24.129 ms 2024-04-25T16:42:28.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 379541 12.83 MB/sec execute 34 sec latency 71.098 ms 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 -- 172.21.15.112:0/3315747369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f10108f00 msgr2=0x7f6f10102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 --2- 172.21.15.112:0/3315747369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f10108f00 0x7f6f10102b50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009900 tx=0x7f6f0402a410 comp rx=0 tx=0).stop 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 -- 172.21.15.112:0/3315747369 shutdown_connections 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 --2- 172.21.15.112:0/3315747369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f101031c0 0x7f6f1010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 --2- 172.21.15.112:0/3315747369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f10108f00 0x7f6f10102b50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 --2- 172.21.15.112:0/3315747369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f10108a60 0x7f6f10102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 -- 172.21.15.112:0/3315747369 >> 172.21.15.112:0/3315747369 conn(0x7f6f100781b0 msgr2=0x7f6f100785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 -- 172.21.15.112:0/3315747369 shutdown_connections 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.090+0000 7f6f15292640 1 -- 172.21.15.112:0/3315747369 wait complete. 2024-04-25T16:42:29.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 Processor -- start 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 -- start start 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 0x7f6f10198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f10108a60 0x7f6f10198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f10108f00 0x7f6f1019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10111bc0 con 0x7f6f10108f00 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10111a40 con 0x7f6f101031c0 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f15292640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f10111d40 con 0x7f6f10108a60 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f0f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f10108f00 0x7f6f1019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.091+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 0x7f6f10198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 0x7f6f10198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:53180/0 (socket says 172.21.15.112:53180) 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f10108a60 0x7f6f10198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0effd640 1 -- 172.21.15.112:0/103417314 learned_addr learned my addr 172.21.15.112:0/103417314 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0f7fe640 1 -- 172.21.15.112:0/103417314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f10108a60 msgr2=0x7f6f10198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0f7fe640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f10108a60 0x7f6f10198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0f7fe640 1 -- 172.21.15.112:0/103417314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 msgr2=0x7f6f10198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0f7fe640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 0x7f6f10198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0f7fe640 1 -- 172.21.15.112:0/103417314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1019d470 con 0x7f6f10108f00 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0effd640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 0x7f6f10198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f0f7fe640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f10108f00 0x7f6f1019ce30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800ec50 tx=0x7f6f0800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6ef7fff640 1 -- 172.21.15.112:0/103417314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08003b70 con 0x7f6f10108f00 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6ef7fff640 1 -- 172.21.15.112:0/103417314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f08003d10 con 0x7f6f10108f00 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1019d760 con 0x7f6f10108f00 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6ef7fff640 1 -- 172.21.15.112:0/103417314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08010830 con 0x7f6f10108f00 2024-04-25T16:42:29.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.092+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1019dc30 con 0x7f6f10108f00 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 Processor -- start 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6ef7fff640 1 -- 172.21.15.112:0/103417314 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f6f080109d0 con 0x7f6f10108f00 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 -- start start 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33c6aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 0x55ad33c6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ad33bd1460 con 0x55ad338aa4d0 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ad33b8b590 con 0x55ad338a22f0 2024-04-25T16:42:29.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f1c213740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad33bcd420 con 0x55ad338cb1b0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 0x55ad33c6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 0x55ad33c6b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56314/0 (socket says 172.21.15.112:56314) 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.095+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3769922316 learned_addr learned my addr 172.21.15.112:0/3769922316 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33c6aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6ef7fff640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6ef80afc50 0x7f6ef80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0f7fe640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6ef80afc50 0x7f6ef80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3769922316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 msgr2=0x55ad33c6aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33c6aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3769922316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 msgr2=0x55ad33c6b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3769922316 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad33b980d0 con 0x55ad338aa4d0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6ef7fff640 1 -- 172.21.15.112:0/103417314 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f6f080df5b0 con 0x7f6f10108f00 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 0x55ad33c6b570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970b6ef9831a2719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:29.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6ef57fa640 1 -- 172.21.15.112:0/3769922316 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04035030 con 0x55ad338aa4d0 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6f0f7fe640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6ef80afc50 0x7f6ef80b2160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f080dd2e0 tx=0x7f6f080ca040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6ef57fa640 1 -- 172.21.15.112:0/3769922316 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f04003d50 con 0x55ad338aa4d0 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.096+0000 7f6ef57fa640 1 -- 172.21.15.112:0/3769922316 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0403dbd0 con 0x55ad338aa4d0 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 -- 172.21.15.112:0/3769922316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 msgr2=0x55ad33c6b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 0x55ad33c6b570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04005780 comp rx=0 tx=0).stop 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 -- 172.21.15.112:0/3769922316 shutdown_connections 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338aa4d0 0x55ad33c6b570 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3769922316 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33c6aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 -- 172.21.15.112:0/3769922316 >> 172.21.15.112:0/3769922316 conn(0x55ad338bf300 msgr2=0x55ad33c672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 -- 172.21.15.112:0/3769922316 shutdown_connections 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.097+0000 7f6f1c213740 1 -- 172.21.15.112:0/3769922316 wait complete. 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 -- start start 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 Processor -- start 2024-04-25T16:42:29.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c76ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ad33c77bc0 con 0x55ad338cb1b0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ad33b58b60 con 0x55ad338a22f0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ad33c77d40 con 0x55ad338aa4d0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f1c213740 10 client.0 _ll_register_callbacks cb 0x55ad33b7d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c750b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56330/0 (socket says 172.21.15.112:56330) 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3636478434 learned_addr learned my addr 172.21.15.112:0/3636478434 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c76ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0effd640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3636478434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 msgr2=0x55ad33c770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c770e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 msgr2=0x55ad33c76ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c76ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.098+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/3636478434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad338e3210 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c750b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005f70 tx=0x7f6f040041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95b28b81a32a818 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/3636478434 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04035070 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/3636478434 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f6f0403db90 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f04003d50 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/3636478434 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6f0403dd90 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 10 client.15723 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ad339af620 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/3636478434 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f04003d50 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/3636478434 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6f04049040 con 0x55ad338cb1b0 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 10 client.15723 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 20 client.15723 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad33889d80 0x55ad33c875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 4 client.15723 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad33bdf070 con 0x55ad33889d80 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.099+0000 7f6f0effd640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad33889d80 0x55ad33c875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.100+0000 7f6ef2ff5640 10 client.15723 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.100+0000 7f6f0effd640 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad33889d80 0x55ad33c875f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ccc831a3a3536d3 server_cookie=62db96caf1a66da in_seq=0 out_seq=0 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/3636478434 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad33bdf070 con 0x55ad33889d80 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6ef2ff5640 10 client.15723 handle_command_reply: tid=0 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 client.15723 shutdown 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 msgr2=0x55ad33c750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c750b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005f70 tx=0x7f6f040041a0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 shutdown_connections 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad33889d80 0x55ad33c875f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c750b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6ef2ff5640 0 client.15723 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 --2- 172.21.15.112:0/3636478434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c76ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 >> 172.21.15.112:0/3636478434 conn(0x55ad338bf300 msgr2=0x55ad33c6c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 shutdown_connections 2024-04-25T16:42:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 1 -- 172.21.15.112:0/3636478434 wait complete. 2024-04-25T16:42:29.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.101+0000 7f6f1c213740 20 client.15723 trim_cache size 0 max 16384 2024-04-25T16:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 Processor -- start 2024-04-25T16:42:29.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 -- start start 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c74860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c8a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101b01c0 con 0x55ad338cb1b0 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ada90 con 0x55ad338a22f0 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f1c213740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ab8e0 con 0x55ad338aa4d0 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c74860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:53208/0 (socket says 172.21.15.112:53208) 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/2575660453 learned_addr learned my addr 172.21.15.112:0/2575660453 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:29.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c8a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/2575660453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 msgr2=0x55ad33c74860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c74860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/2575660453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 msgr2=0x55ad33c8a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c8a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/2575660453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad33c6d380 con 0x55ad338a22f0 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.118+0000 7f6f0effd640 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c74860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.119+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6ce40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f04031f20 tx=0x7f6f0402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7be214ce802d0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.119+0000 7f6ef4ff9640 1 -- 172.21.15.112:0/2575660453 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04035070 con 0x55ad338a22f0 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.119+0000 7f6ef4ff9640 1 -- 172.21.15.112:0/2575660453 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0405b040 con 0x55ad338a22f0 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.119+0000 7f6ef4ff9640 1 -- 172.21.15.112:0/2575660453 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0405a070 con 0x55ad338a22f0 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- 172.21.15.112:0/2575660453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 msgr2=0x55ad33c6ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6ce40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f04031f20 tx=0x7f6f0402a410 comp rx=0 tx=0).stop 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- 172.21.15.112:0/2575660453 shutdown_connections 2024-04-25T16:42:29.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338aa4d0 0x55ad33c74860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338cb1b0 0x55ad33c8a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- 172.21.15.112:0/2575660453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338a22f0 0x55ad33c6ce40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- 172.21.15.112:0/2575660453 >> 172.21.15.112:0/2575660453 conn(0x55ad338bf300 msgr2=0x55ad33c6c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- 172.21.15.112:0/2575660453 shutdown_connections 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- 172.21.15.112:0/2575660453 wait complete. 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- start start 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 Processor -- start 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 0x55ad33c87900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338aa4d0 0x55ad33c87e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33ced710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101abf50 con 0x55ad338a22f0 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ad3c0 con 0x55ad338aa4d0 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6f101ac620 con 0x55ad338cb1b0 2024-04-25T16:42:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f1c213740 10 client.0 _ll_register_callbacks cb 0x7f6f0000b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 0x55ad33c87900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f0effd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338aa4d0 0x55ad33c87e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.120+0000 7f6f0e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 0x55ad33c87900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56360/0 (socket says 172.21.15.112:56360) 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/4172259204 learned_addr learned my addr 172.21.15.112:0/4172259204 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33ced710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/4172259204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 msgr2=0x55ad33ced710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33ced710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338aa4d0 msgr2=0x55ad33c87e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338aa4d0 0x55ad33c87e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 -- 172.21.15.112:0/4172259204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad33cedc50 con 0x55ad338a22f0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0effd640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338aa4d0 0x55ad33c87e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f0e7fc640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 0x55ad33c87900 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009850 tx=0x7f6f04005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e026e3d2b1f1fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6ef2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:42:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/4172259204 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04068060 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f0405b040 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/4172259204 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f6f040314a0 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/4172259204 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6f04035070 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f1c213740 10 client.15729 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.121+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ad33ceedb0 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/4172259204 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0405b040 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/4172259204 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6f04060040 con 0x55ad338a22f0 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f1c213740 10 client.15729 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f1c213740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f1c213740 20 client.15729 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f1c213740 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad3385f1c0 0x55ad33cf1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f1c213740 4 client.15729 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:42:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad33bd8ee0 con 0x55ad3385f1c0 2024-04-25T16:42:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f0effd640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad3385f1c0 0x55ad33cf1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6ef2ff5640 10 client.15729 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.122+0000 7f6f0effd640 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad3385f1c0 0x55ad33cf1290 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceffc776b904b092 server_cookie=da535777c1deac in_seq=0 out_seq=0 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6ef2ff5640 1 -- 172.21.15.112:0/4172259204 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad33bd8ee0 con 0x55ad3385f1c0 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6ef2ff5640 10 client.15729 handle_command_reply: tid=0 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 client.15729 shutdown 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 msgr2=0x55ad33c87900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 0x55ad33c87900 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009850 tx=0x7f6f04005c90 comp rx=0 tx=0).stop 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 shutdown_connections 2024-04-25T16:42:29.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ad3385f1c0 0x55ad33cf1290 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ad338cb1b0 0x55ad33ced710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad338aa4d0 0x55ad33c87e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 --2- 172.21.15.112:0/4172259204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ad338a22f0 0x55ad33c87900 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 >> 172.21.15.112:0/4172259204 conn(0x55ad338bf300 msgr2=0x55ad33c6b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6ef2ff5640 0 client.15729 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 shutdown_connections 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 1 -- 172.21.15.112:0/4172259204 wait complete. 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.268+0000 7f6f1c213740 20 client.15729 trim_cache size 0 max 16384 2024-04-25T16:42:29.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:42:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:42:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:42:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:42:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6ef80afc50 msgr2=0x7f6ef80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6ef80afc50 0x7f6ef80b2160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f080dd2e0 tx=0x7f6f080ca040 comp rx=0 tx=0).stop 2024-04-25T16:42:29.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f10108f00 msgr2=0x7f6f1019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f10108f00 0x7f6f1019ce30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6f0800ec50 tx=0x7f6f0800c6a0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 shutdown_connections 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6ef80afc50 0x7f6ef80b2160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f10108a60 0x7f6f10198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f10108f00 0x7f6f1019ce30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 --2- 172.21.15.112:0/103417314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f101031c0 0x7f6f10198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.269+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 >> 172.21.15.112:0/103417314 conn(0x7f6f100781b0 msgr2=0x7f6f10104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.270+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 shutdown_connections 2024-04-25T16:42:29.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:29.270+0000 7f6f15292640 1 -- 172.21.15.112:0/103417314 wait complete. 2024-04-25T16:42:29.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:29 smithi112 ceph-mon[29122]: pgmap v124: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-04-25T16:42:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:29 smithi133 ceph-mon[32691]: pgmap v124: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-04-25T16:42:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:29 smithi116 ceph-mon[32712]: pgmap v124: 97 pgs: 97 active+clean; 55 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-04-25T16:42:29.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 380821 12.57 MB/sec execute 35 sec latency 82.666 ms 2024-04-25T16:42:30.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 383500 12.69 MB/sec execute 36 sec latency 7.906 ms 2024-04-25T16:42:31.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:31 smithi112 ceph-mon[29122]: pgmap v125: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 433 op/s 2024-04-25T16:42:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:31 smithi133 ceph-mon[32691]: pgmap v125: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 433 op/s 2024-04-25T16:42:31.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:31 smithi116 ceph-mon[32712]: pgmap v125: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 433 op/s 2024-04-25T16:42:31.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 386335 12.71 MB/sec execute 37 sec latency 15.541 ms 2024-04-25T16:42:32.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 388270 12.67 MB/sec execute 38 sec latency 12.992 ms 2024-04-25T16:42:33.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:33 smithi112 ceph-mon[29122]: pgmap v126: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-04-25T16:42:33.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:33 smithi133 ceph-mon[32691]: pgmap v126: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-04-25T16:42:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:33 smithi116 ceph-mon[32712]: pgmap v126: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-04-25T16:42:33.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 390768 12.70 MB/sec execute 39 sec latency 13.954 ms 2024-04-25T16:42:34.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 393425 12.71 MB/sec execute 40 sec latency 39.402 ms 2024-04-25T16:42:35.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:35 smithi112 ceph-mon[29122]: pgmap v127: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-04-25T16:42:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:35 smithi133 ceph-mon[32691]: pgmap v127: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-04-25T16:42:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:35 smithi116 ceph-mon[32712]: pgmap v127: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.4 MiB/s wr, 434 op/s 2024-04-25T16:42:35.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 395376 12.67 MB/sec execute 41 sec latency 12.579 ms 2024-04-25T16:42:36.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 397907 12.70 MB/sec execute 42 sec latency 12.470 ms 2024-04-25T16:42:37.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:37 smithi112 ceph-mon[29122]: pgmap v128: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:42:37.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:42:37.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:42:37.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:42:37.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:42:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:37 smithi133 ceph-mon[32691]: pgmap v128: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:42:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:42:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:42:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:42:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:42:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:37 smithi116 ceph-mon[32712]: pgmap v128: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:42:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:42:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:42:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:42:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:42:37.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 400862 12.81 MB/sec execute 43 sec latency 32.322 ms 2024-04-25T16:42:38.874 INFO:tasks.workunit.client.0.smithi112.stdout: 1 402950 12.69 MB/sec execute 44 sec latency 13.354 ms 2024-04-25T16:42:39.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:39 smithi112 ceph-mon[29122]: pgmap v129: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:42:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:39 smithi133 ceph-mon[32691]: pgmap v129: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:42:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:39 smithi116 ceph-mon[32712]: pgmap v129: 97 pgs: 97 active+clean; 51 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.2 MiB/s wr, 328 op/s 2024-04-25T16:42:39.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 404835 12.71 MB/sec execute 45 sec latency 68.545 ms 2024-04-25T16:42:40.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 407410 12.70 MB/sec execute 46 sec latency 22.493 ms 2024-04-25T16:42:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:41 smithi133 ceph-mon[32691]: pgmap v130: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 435 op/s 2024-04-25T16:42:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:41 smithi116 ceph-mon[32712]: pgmap v130: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 435 op/s 2024-04-25T16:42:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:41 smithi112 ceph-mon[29122]: pgmap v130: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.6 MiB/s wr, 435 op/s 2024-04-25T16:42:41.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 409437 12.68 MB/sec execute 47 sec latency 41.615 ms 2024-04-25T16:42:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 -- 172.21.15.112:0/1406452671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc10d480 msgr2=0x7fb9bc10f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1406452671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc10d480 0x7fb9bc10f8a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009ca0 tx=0x7fb9b402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 -- 172.21.15.112:0/1406452671 shutdown_connections 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1406452671 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc10d480 0x7fb9bc10f8a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1406452671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc103300 0x7fb9bc103710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1406452671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9bc107990 0x7fb9bc102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 -- 172.21.15.112:0/1406452671 >> 172.21.15.112:0/1406452671 conn(0x7fb9bc0fd8a0 msgr2=0x7fb9bc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 -- 172.21.15.112:0/1406452671 shutdown_connections 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 -- 172.21.15.112:0/1406452671 wait complete. 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 Processor -- start 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.372+0000 7fb9c286e640 1 -- start start 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c286e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 0x7fb9bc19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c286e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc107990 0x7fb9bc19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c286e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9bc10d480 0x7fb9bc19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c286e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112c90 con 0x7fb9bc103300 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c286e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112b10 con 0x7fb9bc107990 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c286e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc112e10 con 0x7fb9bc10d480 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 0x7fb9bc19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 0x7fb9bc19b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50528/0 (socket says 172.21.15.112:50528) 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 -- 172.21.15.112:0/1718310933 learned_addr learned my addr 172.21.15.112:0/1718310933 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc107990 0x7fb9bc19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9bc10d480 0x7fb9bc19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 -- 172.21.15.112:0/1718310933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9bc10d480 msgr2=0x7fb9bc19bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9bc10d480 0x7fb9bc19bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 -- 172.21.15.112:0/1718310933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc107990 msgr2=0x7fb9bc19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc107990 0x7fb9bc19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bbfff640 1 -- 172.21.15.112:0/1718310933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9bc1a6870 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.373+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc107990 0x7fb9bc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.374+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 0x7fb9bc19b530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00e9f0 tx=0x7fb9ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.374+0000 7fb9b97fa640 1 -- 172.21.15.112:0/1718310933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac00cd80 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.374+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9bc1a6b60 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.374+0000 7fb9b97fa640 1 -- 172.21.15.112:0/1718310933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9ac0054d0 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.374+0000 7fb9b97fa640 1 -- 172.21.15.112:0/1718310933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac010690 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.374+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9bc1a7030 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.376+0000 7fb9c9801740 1 Processor -- start 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.376+0000 7fb9c9801740 1 -- start start 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e1c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9b97fa640 1 -- 172.21.15.112:0/1718310933 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fb9ac004e90 con 0x7fb9bc103300 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a542f0 0x5563f4e1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e1d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c9801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563f4d83460 con 0x5563f4a5c4d0 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c9801740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5563f4d3d590 con 0x5563f4a542f0 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c9801740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5563f4d7f420 con 0x5563f4a7d1b0 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e1d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e1d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50534/0 (socket says 172.21.15.112:50534) 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/4135098901 learned_addr learned my addr 172.21.15.112:0/4135098901 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a542f0 0x5563f4e1cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e1c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/4135098901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 msgr2=0x5563f4e1c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e1c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/4135098901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a542f0 msgr2=0x5563f4e1cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a542f0 0x5563f4e1cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/4135098901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f4d4a0d0 con 0x5563f4a5c4d0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e1d420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00114f0 tx=0x7fb9b00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacb1c682e4320a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9b97fa640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb9a40afc50 0x7fb9a40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.377+0000 7fb9a27fc640 1 -- 172.21.15.112:0/4135098901 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0005020 con 0x5563f4a5c4d0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9a27fc640 1 -- 172.21.15.112:0/4135098901 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b00047a0 con 0x5563f4a5c4d0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9a27fc640 1 -- 172.21.15.112:0/4135098901 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b000f880 con 0x5563f4a5c4d0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb9a40afc50 0x7fb9a40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9b97fa640 1 -- 172.21.15.112:0/1718310933 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb9ac0df3f0 con 0x7fb9bc103300 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9c9801740 1 -- 172.21.15.112:0/4135098901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 msgr2=0x5563f4e1d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb9a40afc50 0x7fb9a40b2160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4004af0 tx=0x7fb9b4005d80 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.378+0000 7fb9c9801740 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e1d420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00114f0 tx=0x7fb9b00119b0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 -- 172.21.15.112:0/4135098901 shutdown_connections 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e1d420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a542f0 0x5563f4e1cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 --2- 172.21.15.112:0/4135098901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e1c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 -- 172.21.15.112:0/4135098901 >> 172.21.15.112:0/4135098901 conn(0x5563f4a71300 msgr2=0x5563f4e192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 -- 172.21.15.112:0/4135098901 shutdown_connections 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 -- 172.21.15.112:0/4135098901 wait complete. 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 -- start start 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.379+0000 7fb9c9801740 1 Processor -- start 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a542f0 0x5563f4e2b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e31520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5563f4e2e030 con 0x5563f4a5c4d0 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5563f4d0ab60 con 0x5563f4a7d1b0 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5563f4e2e1b0 con 0x5563f4a542f0 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c9801740 10 client.0 _ll_register_callbacks cb 0x5563f4d2f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50540/0 (socket says 172.21.15.112:50540) 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/2954962246 learned_addr learned my addr 172.21.15.112:0/2954962246 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a542f0 0x5563f4e2b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e31520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 -- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 msgr2=0x5563f4e31520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e31520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 -- 172.21.15.112:0/2954962246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 msgr2=0x5563f4e2baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 -- 172.21.15.112:0/2954962246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f4a95210 con 0x5563f4a542f0 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a542f0 0x5563f4e2b5b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0dcb20 tx=0x7fb9ac0e2b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8084e8d72993a285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.380+0000 7fb99b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:42:42.393 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2954962246 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac0a6ec0 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b00047a0 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2954962246 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb9ac0aff40 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2954962246 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb9ac005180 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 10 client.25384 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5563f4b61620 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2954962246 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b00047a0 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2954962246 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb9ac0a2ee0 con 0x5563f4a542f0 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 10 client.25384 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:42.394 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 10 client.25384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 20 client.25384 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a3bd80 0x5563f4e3db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 4 client.25384 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.381+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563f4d91070 con 0x5563f4a3bd80 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.382+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a3bd80 0x5563f4e3db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.382+0000 7fb99b7fe640 10 client.25384 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.382+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a3bd80 0x5563f4e3db30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6463314546e43c29 server_cookie=cd40e6f08af3360c in_seq=0 out_seq=0 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.383+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2954962246 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5563f4d91070 con 0x5563f4a3bd80 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.383+0000 7fb99b7fe640 10 client.25384 handle_command_reply: tid=0 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.383+0000 7fb9c9801740 1 client.25384 shutdown 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a542f0 msgr2=0x5563f4e2b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a542f0 0x5563f4e2b5b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0dcb20 tx=0x7fb9ac0e2b30 comp rx=0 tx=0).stop 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 shutdown_connections 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e31520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.395 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a3bd80 0x5563f4e3db30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2954962246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a542f0 0x5563f4e2b5b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb99b7fe640 0 client.25384 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 >> 172.21.15.112:0/2954962246 conn(0x5563f4a71300 msgr2=0x5563f4e1e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 shutdown_connections 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 1 -- 172.21.15.112:0/2954962246 wait complete. 2024-04-25T16:42:42.396 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.384+0000 7fb9c9801740 20 client.25384 trim_cache size 0 max 16384 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 Processor -- start 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 -- start start 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e31420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1b01b0 con 0x5563f4a542f0 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1ada80 con 0x5563f4a5c4d0 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c9801740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1ab8f0 con 0x5563f4a7d1b0 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e31420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9c0e0b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e31420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50552/0 (socket says 172.21.15.112:50552) 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9bbfff640 1 -- 172.21.15.112:0/2624097309 learned_addr learned my addr 172.21.15.112:0/2624097309 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:42.401 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.400+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9c0e0b640 1 -- 172.21.15.112:0/2624097309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a5c4d0 msgr2=0x5563f4e2a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9c0e0b640 1 -- 172.21.15.112:0/2624097309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 msgr2=0x5563f4e31420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e31420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9c0e0b640 1 -- 172.21.15.112:0/2624097309 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f4e1dcd0 con 0x5563f4a7d1b0 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e31420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e24d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402ade0 tx=0x7fb9b4044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2cb92141a9bde7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9a1ffb640 1 -- 172.21.15.112:0/2624097309 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b402f610 con 0x5563f4a7d1b0 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9a1ffb640 1 -- 172.21.15.112:0/2624097309 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b4036050 con 0x5563f4a7d1b0 2024-04-25T16:42:42.402 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.401+0000 7fb9a1ffb640 1 -- 172.21.15.112:0/2624097309 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4030b50 con 0x5563f4a7d1b0 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 -- 172.21.15.112:0/2624097309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 msgr2=0x5563f4e24d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e24d30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb9b402ade0 tx=0x7fb9b4044000 comp rx=0 tx=0).stop 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 -- 172.21.15.112:0/2624097309 shutdown_connections 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a7d1b0 0x5563f4e24d30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2624097309 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e31420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 -- 172.21.15.112:0/2624097309 >> 172.21.15.112:0/2624097309 conn(0x5563f4a71300 msgr2=0x5563f4e29c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 -- 172.21.15.112:0/2624097309 shutdown_connections 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 -- 172.21.15.112:0/2624097309 wait complete. 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.402+0000 7fb9c9801740 1 -- start start 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 Processor -- start 2024-04-25T16:42:42.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e2a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e2b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1abf40 con 0x5563f4a542f0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1ad3b0 con 0x5563f4a7d1b0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9bc1ac610 con 0x5563f4a5c4d0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c9801740 10 client.0 _ll_register_callbacks cb 0x7fb9ac00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e2a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bb7fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e2a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50560/0 (socket says 172.21.15.112:50560) 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bbfff640 1 -- 172.21.15.112:0/2371751232 learned_addr learned my addr 172.21.15.112:0/2371751232 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9c0e0b640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e2b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 msgr2=0x5563f4e2b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e2b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.408 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/2371751232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 msgr2=0x5563f4e2a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e2a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.403+0000 7fb9bb7fe640 1 -- 172.21.15.112:0/2371751232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f4ea3f70 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e2a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9bb7fe640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2ad70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00114c0 tx=0x7fb9b0013f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d24c68046353c701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb99b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2371751232 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0011050 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2371751232 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fb9b00111f0 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b4036050 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2371751232 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb9b0005100 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 10 client.25390 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5563f4ea51a0 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2371751232 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b4036050 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2371751232 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fb9b0029ce0 con 0x5563f4a5c4d0 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 10 client.25390 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:42.409 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 10 client.25390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 20 client.25390 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a111c0 0x5563f4ea7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 4 client.25390 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.404+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563f4d8aee0 con 0x5563f4a111c0 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.405+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a111c0 0x5563f4ea7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.405+0000 7fb99b7fe640 10 client.25390 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:42.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.405+0000 7fb9bbfff640 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a111c0 0x5563f4ea7710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbc8ee6e9a55c3b server_cookie=dbc66d4111478b5c in_seq=0 out_seq=0 2024-04-25T16:42:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb99b7fe640 1 -- 172.21.15.112:0/2371751232 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563f4d8aee0 con 0x5563f4a111c0 2024-04-25T16:42:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb99b7fe640 10 client.25390 handle_command_reply: tid=0 2024-04-25T16:42:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 client.25390 shutdown 2024-04-25T16:42:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a5c4d0 msgr2=0x5563f4e2ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2ad70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00114c0 tx=0x7fb9b0013f70 comp rx=0 tx=0).stop 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 shutdown_connections 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5563f4a111c0 0x5563f4ea7710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5563f4a5c4d0 0x5563f4e2ad70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.568+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f4a7d1b0 0x5563f4e2b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c9801740 1 --2- 172.21.15.112:0/2371751232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5563f4a542f0 0x5563f4e2a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb99b7fe640 0 client.25390 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 >> 172.21.15.112:0/2371751232 conn(0x5563f4a71300 msgr2=0x5563f4e1d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 shutdown_connections 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c9801740 1 -- 172.21.15.112:0/2371751232 wait complete. 2024-04-25T16:42:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c9801740 20 client.25390 trim_cache size 0 max 16384 2024-04-25T16:42:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:42:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:42:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:42:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:42:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb9a40afc50 msgr2=0x7fb9a40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb9a40afc50 0x7fb9a40b2160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4004af0 tx=0x7fb9b4005d80 comp rx=0 tx=0).stop 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 msgr2=0x7fb9bc19b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.569+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 0x7fb9bc19b530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00e9f0 tx=0x7fb9ac00edf0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 shutdown_connections 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb9a40afc50 0x7fb9a40b2160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9bc10d480 0x7fb9bc19bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9bc107990 0x7fb9bc19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 --2- 172.21.15.112:0/1718310933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9bc103300 0x7fb9bc19b530 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 >> 172.21.15.112:0/1718310933 conn(0x7fb9bc0fd8a0 msgr2=0x7fb9bc10e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 shutdown_connections 2024-04-25T16:42:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:42.570+0000 7fb9c286e640 1 -- 172.21.15.112:0/1718310933 wait complete. 2024-04-25T16:42:42.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 411882 12.71 MB/sec execute 48 sec latency 42.707 ms 2024-04-25T16:42:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:43 smithi133 ceph-mon[32691]: pgmap v131: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-04-25T16:42:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:43 smithi116 ceph-mon[32712]: pgmap v131: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-04-25T16:42:43.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:43 smithi112 ceph-mon[29122]: pgmap v131: 97 pgs: 97 active+clean; 58 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 325 op/s 2024-04-25T16:42:43.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 414687 12.72 MB/sec execute 49 sec latency 13.207 ms 2024-04-25T16:42:44.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 416113 12.62 MB/sec execute 50 sec latency 95.020 ms 2024-04-25T16:42:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:45 smithi133 ceph-mon[32691]: pgmap v132: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 549 op/s 2024-04-25T16:42:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:45 smithi116 ceph-mon[32712]: pgmap v132: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 549 op/s 2024-04-25T16:42:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:45 smithi112 ceph-mon[29122]: pgmap v132: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.6 MiB/s wr, 549 op/s 2024-04-25T16:42:45.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 418895 12.70 MB/sec execute 51 sec latency 19.179 ms 2024-04-25T16:42:46.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 421511 12.71 MB/sec execute 52 sec latency 12.764 ms 2024-04-25T16:42:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:47 smithi133 ceph-mon[32691]: pgmap v133: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-04-25T16:42:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:47 smithi116 ceph-mon[32712]: pgmap v133: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-04-25T16:42:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:47 smithi112 ceph-mon[29122]: pgmap v133: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-04-25T16:42:47.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 423557 12.69 MB/sec execute 53 sec latency 17.638 ms 2024-04-25T16:42:48.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 426138 12.71 MB/sec execute 54 sec latency 15.178 ms 2024-04-25T16:42:49.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:49 smithi133 ceph-mon[32691]: pgmap v134: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-04-25T16:42:49.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:49 smithi116 ceph-mon[32712]: pgmap v134: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-04-25T16:42:49.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:49 smithi112 ceph-mon[29122]: pgmap v134: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 331 op/s 2024-04-25T16:42:49.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 428574 12.70 MB/sec execute 55 sec latency 102.959 ms 2024-04-25T16:42:50.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 430401 12.64 MB/sec execute 56 sec latency 19.934 ms 2024-04-25T16:42:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:51 smithi133 ceph-mon[32691]: pgmap v135: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 443 op/s 2024-04-25T16:42:51.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:51 smithi116 ceph-mon[32712]: pgmap v135: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 443 op/s 2024-04-25T16:42:51.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:51 smithi112 ceph-mon[29122]: pgmap v135: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 4.5 MiB/s wr, 443 op/s 2024-04-25T16:42:51.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 433175 12.71 MB/sec execute 57 sec latency 13.514 ms 2024-04-25T16:42:52.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 436114 12.74 MB/sec execute 58 sec latency 14.153 ms 2024-04-25T16:42:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:53 smithi133 ceph-mon[32691]: pgmap v136: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 336 op/s 2024-04-25T16:42:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:53 smithi116 ceph-mon[32712]: pgmap v136: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 336 op/s 2024-04-25T16:42:53.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:53 smithi112 ceph-mon[29122]: pgmap v136: 97 pgs: 97 active+clean; 53 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 336 op/s 2024-04-25T16:42:53.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 438605 12.72 MB/sec execute 59 sec latency 13.770 ms 2024-04-25T16:42:54.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 440309 12.72 MB/sec execute 60 sec latency 128.676 ms 2024-04-25T16:42:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:55 smithi133 ceph-mon[32691]: pgmap v137: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 446 op/s 2024-04-25T16:42:55.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:55 smithi116 ceph-mon[32712]: pgmap v137: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 446 op/s 2024-04-25T16:42:55.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:55 smithi112 ceph-mon[29122]: pgmap v137: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.7 MiB/s wr, 446 op/s 2024-04-25T16:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2768246852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410e5a0 msgr2=0x7f80b41109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2768246852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410e5a0 0x7f80b41109c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009ca0 tx=0x7f80ac02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2768246852 shutdown_connections 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2768246852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b41020a0 0x7f80b41024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2768246852 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b410e5a0 0x7f80b41109c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2768246852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80b4109bf0 0x7f80b4101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2768246852 >> 172.21.15.112:0/2768246852 conn(0x7f80b40778b0 msgr2=0x7f80b4077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2768246852 shutdown_connections 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2768246852 wait complete. 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 Processor -- start 2024-04-25T16:42:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.673+0000 7f80bc2aa640 1 -- start start 2024-04-25T16:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 0x7f80b419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80b4109bf0 0x7f80b419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b410e5a0 0x7f80b419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4113d60 con 0x7f80b41020a0 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4113be0 con 0x7f80b4109bf0 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80b4113ee0 con 0x7f80b410e5a0 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 0x7f80b419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 0x7f80b419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42880/0 (socket says 172.21.15.112:42880) 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 -- 172.21.15.112:0/2071877719 learned_addr learned my addr 172.21.15.112:0/2071877719 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80b9845640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80b4109bf0 0x7f80b419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba847640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b410e5a0 0x7f80b419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 -- 172.21.15.112:0/2071877719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b410e5a0 msgr2=0x7f80b419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b410e5a0 0x7f80b419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 -- 172.21.15.112:0/2071877719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80b4109bf0 msgr2=0x7f80b419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80b4109bf0 0x7f80b419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 -- 172.21.15.112:0/2071877719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b419f5a0 con 0x7f80b41020a0 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ba046640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 0x7f80b419a740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e9f0 tx=0x7f80b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ab7fe640 1 -- 172.21.15.112:0/2071877719 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b000ccc0 con 0x7f80b41020a0 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b419f890 con 0x7f80b41020a0 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ab7fe640 1 -- 172.21.15.112:0/2071877719 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b00035f0 con 0x7f80b41020a0 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80ab7fe640 1 -- 172.21.15.112:0/2071877719 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b000ce60 con 0x7f80b41020a0 2024-04-25T16:42:55.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.674+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b41a7210 con 0x7f80b41020a0 2024-04-25T16:42:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.677+0000 7f80ab7fe640 1 -- 172.21.15.112:0/2071877719 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f80b0003aa0 con 0x7f80b41020a0 2024-04-25T16:42:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.677+0000 7f80c3233740 1 Processor -- start 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 -- start start 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19e21b0 0x55c0b1d81980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d81ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19c14d0 0x55c0b1d82400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c0b1ce8460 con 0x55c0b19b92f0 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c0b1ca2590 con 0x55c0b19e21b0 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80c3233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c0b1ce4420 con 0x55c0b19c14d0 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d81ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ba847640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19e21b0 0x55c0b1d81980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80b9845640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19c14d0 0x55c0b1d82400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ba847640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19e21b0 0x55c0b1d81980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:60736/0 (socket says 172.21.15.112:60736) 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d81ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42888/0 (socket says 172.21.15.112:42888) 2024-04-25T16:42:55.684 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ba847640 1 -- 172.21.15.112:0/1167563233 learned_addr learned my addr 172.21.15.112:0/1167563233 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ab7fe640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f80a00afc50 0x7f80a00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80b9845640 1 -- 172.21.15.112:0/1167563233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19e21b0 msgr2=0x55c0b1d81980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80b9845640 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19e21b0 0x55c0b1d81980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80b9845640 1 -- 172.21.15.112:0/1167563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 msgr2=0x55c0b1d81ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80b9845640 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d81ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80b9845640 1 -- 172.21.15.112:0/1167563233 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b1caf0d0 con 0x55c0b19c14d0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ba847640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f80a00afc50 0x7f80a00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.678+0000 7f80ab7fe640 1 -- 172.21.15.112:0/2071877719 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f80b00df440 con 0x7f80b41020a0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.679+0000 7f80b9845640 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19c14d0 0x55c0b1d82400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80b40756d0 tx=0x7f80a4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3020a8fd1feb04cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.679+0000 7f80a8bfe640 1 -- 172.21.15.112:0/1167563233 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4004230 con 0x55c0b19c14d0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.679+0000 7f80ba847640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f80a00afc50 0x7f80a00b2160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0094e0 tx=0x7f80ac0095c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.679+0000 7f80a8bfe640 1 -- 172.21.15.112:0/1167563233 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a4009e10 con 0x55c0b19c14d0 2024-04-25T16:42:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.679+0000 7f80a8bfe640 1 -- 172.21.15.112:0/1167563233 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4004c60 con 0x55c0b19c14d0 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 -- 172.21.15.112:0/1167563233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19c14d0 msgr2=0x55c0b1d82400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19c14d0 0x55c0b1d82400 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f80b40756d0 tx=0x7f80a4011810 comp rx=0 tx=0).stop 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 -- 172.21.15.112:0/1167563233 shutdown_connections 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19c14d0 0x55c0b1d82400 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d81ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 --2- 172.21.15.112:0/1167563233 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19e21b0 0x55c0b1d81980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 -- 172.21.15.112:0/1167563233 >> 172.21.15.112:0/1167563233 conn(0x55c0b19d6300 msgr2=0x55c0b1d7e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 -- 172.21.15.112:0/1167563233 shutdown_connections 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 -- 172.21.15.112:0/1167563233 wait complete. 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 -- start start 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.680+0000 7f80c3233740 1 Processor -- start 2024-04-25T16:42:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 0x55c0b1d905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d90ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19e21b0 0x55c0b1d96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c0b1d93090 con 0x55c0b19e21b0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c0b1c6fb60 con 0x55c0b19c14d0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c0b1d93210 con 0x55c0b19b92f0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80c3233740 10 client.0 _ll_register_callbacks cb 0x55c0b1c942b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba847640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19e21b0 0x55c0b1d96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 0x55c0b1d905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80b9845640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d90ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 0x55c0b1d905a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:39438/0 (socket says 172.21.15.112:39438) 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 -- 172.21.15.112:0/688070345 learned_addr learned my addr 172.21.15.112:0/688070345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80b9845640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d90ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:60738/0 (socket says 172.21.15.112:60738) 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 -- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 msgr2=0x55c0b1d90ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d90ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 -- 172.21.15.112:0/688070345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19e21b0 msgr2=0x55c0b1d96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19e21b0 0x55c0b1d96580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 -- 172.21.15.112:0/688070345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b19fa210 con 0x55c0b19b92f0 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba847640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19e21b0 0x55c0b1d96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80b9845640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f80ba046640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 0x55c0b1d905a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f80b00dd170 tx=0x7f80b00e2b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=950f23f8d5cb356b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.681+0000 7f809dffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f809dffb640 1 -- 172.21.15.112:0/688070345 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a6f10 con 0x55c0b19b92f0 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80a4009e10 con 0x55c0b19b92f0 2024-04-25T16:42:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f809dffb640 1 -- 172.21.15.112:0/688070345 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f80b00aff90 con 0x55c0b19b92f0 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f809dffb640 1 -- 172.21.15.112:0/688070345 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f80b00a6d30 con 0x55c0b19b92f0 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 10 client.25399 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c0b1ac6620 con 0x55c0b19b92f0 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f809dffb640 1 -- 172.21.15.112:0/688070345 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80a4009e10 con 0x55c0b19b92f0 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f809dffb640 1 -- 172.21.15.112:0/688070345 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f80b00a2f30 con 0x55c0b19b92f0 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 10 client.25399 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 20 client.25399 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19a0d80 0x55c0b1da2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 4 client.25399 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b1cf6070 con 0x55c0b19a0d80 2024-04-25T16:42:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.682+0000 7f80b9845640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19a0d80 0x55c0b1da2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.683+0000 7f809dffb640 10 client.25399 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.683+0000 7f80b9845640 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19a0d80 0x55c0b1da2b90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27b31c56217dbfb server_cookie=93c0a699caf3599e in_seq=0 out_seq=0 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.684+0000 7f809dffb640 1 -- 172.21.15.112:0/688070345 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0b1cf6070 con 0x55c0b19a0d80 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.684+0000 7f809dffb640 10 client.25399 handle_command_reply: tid=0 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.684+0000 7f80c3233740 1 client.25399 shutdown 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 msgr2=0x55c0b1d905a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 0x55c0b1d905a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f80b00dd170 tx=0x7f80b00e2b80 comp rx=0 tx=0).stop 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 shutdown_connections 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19e21b0 0x55c0b1d96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d90ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19a0d80 0x55c0b1da2b90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 --2- 172.21.15.112:0/688070345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19b92f0 0x55c0b1d905a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f809dffb640 0 client.25399 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 >> 172.21.15.112:0/688070345 conn(0x55c0b19d6300 msgr2=0x55c0b1d83810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 shutdown_connections 2024-04-25T16:42:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 1 -- 172.21.15.112:0/688070345 wait complete. 2024-04-25T16:42:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.685+0000 7f80c3233740 20 client.25399 trim_cache size 0 max 16384 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 Processor -- start 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 -- start start 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d96440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d89d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41b0160 con 0x55c0b19b92f0 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41ada30 con 0x55c0b19c14d0 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.700+0000 7f80c3233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41ab880 con 0x55c0b19e21b0 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d96440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d96440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42906/0 (socket says 172.21.15.112:42906) 2024-04-25T16:42:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 -- 172.21.15.112:0/366187727 learned_addr learned my addr 172.21.15.112:0/366187727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba847640 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80b9845640 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d89d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 -- 172.21.15.112:0/366187727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 msgr2=0x55c0b1d8a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d8a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 -- 172.21.15.112:0/366187727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 msgr2=0x55c0b1d89d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d89d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 -- 172.21.15.112:0/366187727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b1d79a60 con 0x55c0b19b92f0 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f80ba046640 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d96440 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f80b00a4830 tx=0x7f80b0003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f076d9e653e7edb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f809ffff640 1 -- 172.21.15.112:0/366187727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00a6f10 con 0x55c0b19b92f0 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f809ffff640 1 -- 172.21.15.112:0/366187727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b00a70d0 con 0x55c0b19b92f0 2024-04-25T16:42:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.701+0000 7f809ffff640 1 -- 172.21.15.112:0/366187727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00900c0 con 0x55c0b19b92f0 2024-04-25T16:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 -- 172.21.15.112:0/366187727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 msgr2=0x55c0b1d96440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d96440 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f80b00a4830 tx=0x7f80b0003e70 comp rx=0 tx=0).stop 2024-04-25T16:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 -- 172.21.15.112:0/366187727 shutdown_connections 2024-04-25T16:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d89d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.705 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 --2- 172.21.15.112:0/366187727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d96440 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 -- 172.21.15.112:0/366187727 >> 172.21.15.112:0/366187727 conn(0x55c0b19d6300 msgr2=0x55c0b1d822b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 -- 172.21.15.112:0/366187727 shutdown_connections 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 -- 172.21.15.112:0/366187727 wait complete. 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.702+0000 7f80c3233740 1 -- start start 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 Processor -- start 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d97d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d98280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41abef0 con 0x55c0b19b92f0 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41ad360 con 0x55c0b19c14d0 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80b41ac5c0 con 0x55c0b19e21b0 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80c3233740 10 client.0 _ll_register_callbacks cb 0x7f80b000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d97d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d97d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42922/0 (socket says 172.21.15.112:42922) 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 -- 172.21.15.112:0/3625159743 learned_addr learned my addr 172.21.15.112:0/3625159743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80b9845640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d98280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba847640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 -- 172.21.15.112:0/3625159743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 msgr2=0x55c0b1d987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 -- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 msgr2=0x55c0b1d98280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d98280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 -- 172.21.15.112:0/3625159743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b1e09160 con 0x55c0b19b92f0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f80ba046640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d97d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000c00 tx=0x7f80b00a3160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f309c4d640b45bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.703+0000 7f809dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f809dffb640 1 -- 172.21.15.112:0/3625159743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b00ab970 con 0x55c0b19b92f0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f809dffb640 1 -- 172.21.15.112:0/3625159743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f80b00a6a30 con 0x55c0b19b92f0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80b00a70d0 con 0x55c0b19b92f0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f809dffb640 1 -- 172.21.15.112:0/3625159743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f80b00a6f10 con 0x55c0b19b92f0 2024-04-25T16:42:55.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 10 client.15765 fetch_fsmap learned FSMap version 23 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c0b1e0a3f0 con 0x55c0b19b92f0 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f809dffb640 1 -- 172.21.15.112:0/3625159743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80b00a70d0 con 0x55c0b19b92f0 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f809dffb640 1 -- 172.21.15.112:0/3625159743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f80b00a6290 con 0x55c0b19b92f0 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 10 client.15765 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 20 client.15765 populate_metadata read hostname 'smithi112' 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19761c0 0x55c0b1e0c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 4 client.15765 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0b1cefee0 con 0x55c0b19761c0 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.704+0000 7f80b9845640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19761c0 0x55c0b1e0c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.705+0000 7f80b9845640 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19761c0 0x55c0b1e0c8d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39551d6320a5275f server_cookie=c87a0f65692cc1f8 in_seq=0 out_seq=0 2024-04-25T16:42:55.708 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.705+0000 7f809dffb640 10 client.15765 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.855+0000 7f809dffb640 1 -- 172.21.15.112:0/3625159743 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0b1cefee0 con 0x55c0b19761c0 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f809dffb640 10 client.15765 handle_command_reply: tid=0 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 client.15765 shutdown 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 msgr2=0x55c0b1d97d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d97d40 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f80b0000c00 tx=0x7f80b00a3160 comp rx=0 tx=0).stop 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 shutdown_connections 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c0b19761c0 0x55c0b1e0c8d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c0b19e21b0 0x55c0b1d987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c0b19c14d0 0x55c0b1d98280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 --2- 172.21.15.112:0/3625159743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c0b19b92f0 0x55c0b1d97d40 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 >> 172.21.15.112:0/3625159743 conn(0x55c0b19d6300 msgr2=0x55c0b1d82010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f809dffb640 0 client.15765 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 shutdown_connections 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 1 -- 172.21.15.112:0/3625159743 wait complete. 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.856+0000 7f80c3233740 20 client.15765 trim_cache size 0 max 16384 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:42:55.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f80a00afc50 msgr2=0x7f80a00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f80a00afc50 0x7f80a00b2160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80ac0094e0 tx=0x7f80ac0095c0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 msgr2=0x7f80b419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 0x7f80b419a740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f80b000e9f0 tx=0x7f80b000eeb0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 shutdown_connections 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f80a00afc50 0x7f80a00b2160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b410e5a0 0x7f80b419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80b4109bf0 0x7f80b419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 --2- 172.21.15.112:0/2071877719 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80b41020a0 0x7f80b419a740 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:42:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 >> 172.21.15.112:0/2071877719 conn(0x7f80b40778b0 msgr2=0x7f80b410a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:42:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 shutdown_connections 2024-04-25T16:42:55.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:42:55.857+0000 7f80bc2aa640 1 -- 172.21.15.112:0/2071877719 wait complete. 2024-04-25T16:42:55.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 442451 12.64 MB/sec execute 61 sec latency 80.103 ms 2024-04-25T16:42:56.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 444540 12.65 MB/sec execute 62 sec latency 13.832 ms 2024-04-25T16:42:57.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:57 smithi133 ceph-mon[32691]: pgmap v138: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-04-25T16:42:57.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:57 smithi116 ceph-mon[32712]: pgmap v138: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-04-25T16:42:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:57 smithi112 ceph-mon[29122]: pgmap v138: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-04-25T16:42:57.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 447334 12.72 MB/sec execute 63 sec latency 13.478 ms 2024-04-25T16:42:58.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 450120 12.72 MB/sec execute 64 sec latency 12.185 ms 2024-04-25T16:42:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:42:59 smithi133 ceph-mon[32691]: pgmap v139: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-04-25T16:42:59.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:42:59 smithi116 ceph-mon[32712]: pgmap v139: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-04-25T16:42:59.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:42:59 smithi112 ceph-mon[29122]: pgmap v139: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 222 op/s 2024-04-25T16:42:59.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 451534 12.65 MB/sec execute 65 sec latency 96.066 ms 2024-04-25T16:43:00.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 453516 12.62 MB/sec execute 66 sec latency 24.771 ms 2024-04-25T16:43:01.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:01 smithi133 ceph-mon[32691]: pgmap v140: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 451 op/s 2024-04-25T16:43:01.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:01 smithi116 ceph-mon[32712]: pgmap v140: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 451 op/s 2024-04-25T16:43:01.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:01 smithi112 ceph-mon[29122]: pgmap v140: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 451 op/s 2024-04-25T16:43:01.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 455440 12.60 MB/sec execute 67 sec latency 18.173 ms 2024-04-25T16:43:02.875 INFO:tasks.workunit.client.0.smithi112.stdout: 1 457974 12.63 MB/sec execute 68 sec latency 13.448 ms 2024-04-25T16:43:03.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:03 smithi133 ceph-mon[32691]: pgmap v141: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-04-25T16:43:03.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:03 smithi116 ceph-mon[32712]: pgmap v141: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-04-25T16:43:03.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:03 smithi112 ceph-mon[29122]: pgmap v141: 97 pgs: 97 active+clean; 49 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 339 op/s 2024-04-25T16:43:03.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 459621 12.78 MB/sec execute 69 sec latency 51.472 ms 2024-04-25T16:43:04.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 462303 12.84 MB/sec execute 70 sec latency 42.907 ms 2024-04-25T16:43:05.397 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:05 smithi112 ceph-mon[29122]: pgmap v142: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 459 op/s 2024-04-25T16:43:05.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:05 smithi133 ceph-mon[32691]: pgmap v142: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 459 op/s 2024-04-25T16:43:05.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:05 smithi116 ceph-mon[32712]: pgmap v142: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.8 MiB/s wr, 459 op/s 2024-04-25T16:43:05.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 464881 12.84 MB/sec execute 71 sec latency 23.077 ms 2024-04-25T16:43:06.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 467105 12.82 MB/sec execute 72 sec latency 13.532 ms 2024-04-25T16:43:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:07 smithi133 ceph-mon[32691]: pgmap v143: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-04-25T16:43:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:07 smithi116 ceph-mon[32712]: pgmap v143: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-04-25T16:43:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:07 smithi112 ceph-mon[29122]: pgmap v143: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-04-25T16:43:07.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 12.84 MB/sec execute 73 sec latency 53.830 ms 2024-04-25T16:43:08.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 12.66 MB/sec execute 74 sec latency 1053.868 ms 2024-04-25T16:43:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/3627109629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 msgr2=0x7f6a38101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/3627109629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a38101b60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009900 tx=0x7f6a3002a480 comp rx=0 tx=0).stop 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/3627109629 shutdown_connections 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/3627109629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 0x7f6a38107700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/3627109629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38108e60 0x7f6a381020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/3627109629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a38101b60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/3627109629 >> 172.21.15.112:0/3627109629 conn(0x7f6a380778b0 msgr2=0x7f6a38077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/3627109629 shutdown_connections 2024-04-25T16:43:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.967+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/3627109629 wait complete. 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 Processor -- start 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 -- start start 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 0x7f6a381a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a381a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38108e60 0x7f6a381a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3811a350 con 0x7f6a38108a50 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3811a1d0 con 0x7f6a38108e60 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3fbc5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3811a4d0 con 0x7f6a381072f0 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a381a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a381a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40018/0 (socket says 172.21.15.112:40018) 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 -- 172.21.15.112:0/945324829 learned_addr learned my addr 172.21.15.112:0/945324829 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:08.969 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3e162640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38108e60 0x7f6a381a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d961640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 0x7f6a381a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 -- 172.21.15.112:0/945324829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 msgr2=0x7f6a381a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 0x7f6a381a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 -- 172.21.15.112:0/945324829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38108e60 msgr2=0x7f6a381a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38108e60 0x7f6a381a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d160640 1 -- 172.21.15.112:0/945324829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381a5c10 con 0x7f6a38108a50 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.968+0000 7f6a3d961640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 0x7f6a381a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.969+0000 7f6a3d160640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a381a1310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00d770 tx=0x7f6a2c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.969+0000 7f6a36ffd640 1 -- 172.21.15.112:0/945324829 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c017070 con 0x7f6a38108a50 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.969+0000 7f6a36ffd640 1 -- 172.21.15.112:0/945324829 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2c009c30 con 0x7f6a38108a50 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.969+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a381a5f00 con 0x7f6a38108a50 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.969+0000 7f6a36ffd640 1 -- 172.21.15.112:0/945324829 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00f620 con 0x7f6a38108a50 2024-04-25T16:43:08.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.969+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381ad880 con 0x7f6a38108a50 2024-04-25T16:43:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a36ffd640 1 -- 172.21.15.112:0/945324829 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f6a2c01a070 con 0x7f6a38108a50 2024-04-25T16:43:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 Processor -- start 2024-04-25T16:43:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 -- start start 2024-04-25T16:43:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 0x5590129d3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559012939460 con 0x5590126124d0 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5590128f3590 con 0x55901260a2f0 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.972+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559012935420 con 0x5590126331b0 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 0x5590129d3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 0x5590129d3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40022/0 (socket says 172.21.15.112:40022) 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3e162640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 -- 172.21.15.112:0/922117564 learned_addr learned my addr 172.21.15.112:0/922117564 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d961640 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a36ffd640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6a200afc50 0x7f6a200b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 -- 172.21.15.112:0/922117564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 msgr2=0x5590129d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129d2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 -- 172.21.15.112:0/922117564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 msgr2=0x5590129d3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129d3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 -- 172.21.15.112:0/922117564 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590129000d0 con 0x5590126124d0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3e162640 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d961640 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3e162640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6a200afc50 0x7f6a200b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a36ffd640 1 -- 172.21.15.112:0/945324829 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f6a2c0deb00 con 0x7f6a38108a50 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.973+0000 7f6a3d160640 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 0x5590129d3570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0dc1a0 tx=0x7f6a2c061a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2562abc04ca48a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.974+0000 7f6a1ffff640 1 -- 172.21.15.112:0/922117564 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0a6a50 con 0x5590126124d0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.974+0000 7f6a3e162640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6a200afc50 0x7f6a200b2160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009fd0 tx=0x7f6a28004a40 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.974+0000 7f6a1ffff640 1 -- 172.21.15.112:0/922117564 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2c0a6790 con 0x5590126124d0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.974+0000 7f6a1ffff640 1 -- 172.21.15.112:0/922117564 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007420 con 0x5590126124d0 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.974+0000 7f6a46a7a740 1 -- 172.21.15.112:0/922117564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 msgr2=0x5590129d3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.974+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 0x5590129d3570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0dc1a0 tx=0x7f6a2c061a50 comp rx=0 tx=0).stop 2024-04-25T16:43:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 -- 172.21.15.112:0/922117564 shutdown_connections 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590126124d0 0x5590129d3570 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129d3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/922117564 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 -- 172.21.15.112:0/922117564 >> 172.21.15.112:0/922117564 conn(0x559012627300 msgr2=0x5590129cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 -- 172.21.15.112:0/922117564 shutdown_connections 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 -- 172.21.15.112:0/922117564 wait complete. 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 -- start start 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.975+0000 7f6a46a7a740 1 Processor -- start 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129deb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5590129dfbc0 con 0x55901260a2f0 2024-04-25T16:43:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5590128c0b60 con 0x5590126124d0 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5590129dfd40 con 0x5590126331b0 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 10 client.0 _ll_register_callbacks cb 0x5590128e52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129deb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129deb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40026/0 (socket says 172.21.15.112:40026) 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 -- 172.21.15.112:0/3728591314 learned_addr learned my addr 172.21.15.112:0/3728591314 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3e162640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 -- 172.21.15.112:0/3728591314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 msgr2=0x5590129dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129dcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 -- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 msgr2=0x5590129df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 -- 172.21.15.112:0/3728591314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55901264b210 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3e162640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a3d961640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129deb10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6a3806a0a0 tx=0x7f6a30005f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3014691458a616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3728591314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a30040070 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3728591314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f6a30004240 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.976+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2c0a6790 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3728591314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a30035040 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 10 client.15777 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559012717620 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3728591314 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2c0a6790 con 0x55901260a2f0 2024-04-25T16:43:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3728591314 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6a30004910 con 0x55901260a2f0 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 10 client.15777 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 20 client.15777 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590125f1d80 0x5590129ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 4 client.15777 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559012947070 con 0x5590125f1d80 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.977+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590125f1d80 0x5590129ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.978+0000 7f6a1d7fa640 10 client.15777 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.978+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590125f1d80 0x5590129ef6e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd2266a9c68f2c2 server_cookie=3e6f5810064067f7 in_seq=0 out_seq=0 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.978+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3728591314 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559012947070 con 0x5590125f1d80 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.978+0000 7f6a1d7fa640 10 client.15777 handle_command_reply: tid=0 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 client.15777 shutdown 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 msgr2=0x5590129deb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129deb10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6a3806a0a0 tx=0x7f6a30005f20 comp rx=0 tx=0).stop 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 shutdown_connections 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590125f1d80 0x5590129ef6e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126331b0 0x5590129dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a1d7fa640 0 client.15777 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3728591314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129deb10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 >> 172.21.15.112:0/3728591314 conn(0x559012627300 msgr2=0x5590129d4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 shutdown_connections 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3728591314 wait complete. 2024-04-25T16:43:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.979+0000 7f6a46a7a740 20 client.15777 trim_cache size 0 max 16384 2024-04-25T16:43:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 Processor -- start 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 -- start start 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590125f1d80 0x5590129deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129d6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b8b40 con 0x55901260a2f0 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b6410 con 0x5590126124d0 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b4260 con 0x5590125f1d80 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a3d160640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a3d160640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129d6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40034/0 (socket says 172.21.15.112:40034) 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.994+0000 7f6a3d160640 1 -- 172.21.15.112:0/2370241550 learned_addr learned my addr 172.21.15.112:0/2370241550 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d961640 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590125f1d80 0x5590129deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3e162640 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129d6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d160640 1 -- 172.21.15.112:0/2370241550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590125f1d80 msgr2=0x5590129deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d160640 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590125f1d80 0x5590129deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d160640 1 -- 172.21.15.112:0/2370241550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 msgr2=0x5590129d6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d160640 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129d6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d160640 1 -- 172.21.15.112:0/2370241550 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590129e9e80 con 0x55901260a2f0 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a3d160640 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129d6870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c000c00 tx=0x7f6a2c0a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eba8715752357a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a1f7fe640 1 -- 172.21.15.112:0/2370241550 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c021040 con 0x55901260a2f0 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a1f7fe640 1 -- 172.21.15.112:0/2370241550 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2c0a6790 con 0x55901260a2f0 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.995+0000 7f6a1f7fe640 1 -- 172.21.15.112:0/2370241550 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0a63f0 con 0x55901260a2f0 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 -- 172.21.15.112:0/2370241550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 msgr2=0x5590129d6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129d6870 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c000c00 tx=0x7f6a2c0a1040 comp rx=0 tx=0).stop 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 -- 172.21.15.112:0/2370241550 shutdown_connections 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590126124d0 0x5590129d6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55901260a2f0 0x5590129d6870 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/2370241550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590125f1d80 0x5590129deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 -- 172.21.15.112:0/2370241550 >> 172.21.15.112:0/2370241550 conn(0x559012627300 msgr2=0x5590129de470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 -- 172.21.15.112:0/2370241550 shutdown_connections 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 -- 172.21.15.112:0/2370241550 wait complete. 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 -- start start 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.996+0000 7f6a46a7a740 1 Processor -- start 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 0x5590129e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126124d0 0x5590129e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b48d0 con 0x5590125f1d80 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b5d40 con 0x55901260a2f0 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a381b4fa0 con 0x5590126124d0 2024-04-25T16:43:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a46a7a740 10 client.0 _ll_register_callbacks cb 0x7f6a30005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 0x5590129e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 0x5590129e56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40036/0 (socket says 172.21.15.112:40036) 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 -- 172.21.15.112:0/3246462139 learned_addr learned my addr 172.21.15.112:0/3246462139 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3e162640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126124d0 0x5590129e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 -- 172.21.15.112:0/3246462139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126124d0 msgr2=0x5590129e30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126124d0 0x5590129e30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 -- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 msgr2=0x5590129e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 -- 172.21.15.112:0/3246462139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590129e3820 con 0x5590125f1d80 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a3d961640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 0x5590129e56d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009850 tx=0x7f6a30031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350f7d35c38d8d56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:43:09.001 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.997+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3246462139 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3005a060 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2c0a6790 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3246462139 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f6a30004f50 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3246462139 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a30050030 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 10 client.15783 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x559012a57b00 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3246462139 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a2c0a6790 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3246462139 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f6a3003c7b0 con 0x5590125f1d80 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 10 client.15783 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 20 client.15783 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590126331b0 0x559012a5a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 4 client.15783 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559012940ee0 con 0x5590126331b0 2024-04-25T16:43:09.002 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.998+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590126331b0 0x559012a5a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.999+0000 7f6a1d7fa640 10 client.15783 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:09.003 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:08.999+0000 7f6a3d160640 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590126331b0 0x559012a5a2c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b44521fd7c8880f server_cookie=cd034686f00cdace in_seq=0 out_seq=0 2024-04-25T16:43:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:09 smithi133 ceph-mon[32691]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-04-25T16:43:09.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:09 smithi116 ceph-mon[32712]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-04-25T16:43:09.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:09 smithi112 ceph-mon[29122]: pgmap v144: 97 pgs: 97 active+clean; 55 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-04-25T16:43:09.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 12.49 MB/sec execute 75 sec latency 2053.983 ms 2024-04-25T16:43:10.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 12.33 MB/sec execute 76 sec latency 3054.081 ms 2024-04-25T16:43:11.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:11 smithi133 ceph-mon[32691]: pgmap v145: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 448 op/s 2024-04-25T16:43:11.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:11 smithi116 ceph-mon[32712]: pgmap v145: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 448 op/s 2024-04-25T16:43:11.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:11 smithi112 ceph-mon[29122]: pgmap v145: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.8 MiB/s wr, 448 op/s 2024-04-25T16:43:11.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 12.17 MB/sec execute 77 sec latency 4054.181 ms 2024-04-25T16:43:12.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 12.01 MB/sec execute 78 sec latency 5054.290 ms 2024-04-25T16:43:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:13 smithi133 ceph-mon[32691]: pgmap v146: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 220 op/s 2024-04-25T16:43:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:13 smithi116 ceph-mon[32712]: pgmap v146: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 220 op/s 2024-04-25T16:43:13.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:13 smithi112 ceph-mon[29122]: pgmap v146: 97 pgs: 97 active+clean; 61 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 220 op/s 2024-04-25T16:43:13.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 11.86 MB/sec execute 79 sec latency 6054.384 ms 2024-04-25T16:43:14.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 469586 11.71 MB/sec execute 80 sec latency 7054.416 ms 2024-04-25T16:43:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.347+0000 7f6a1d7fa640 1 -- 172.21.15.112:0/3246462139 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559012940ee0 con 0x5590126331b0 2024-04-25T16:43:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.347+0000 7f6a1d7fa640 10 client.15783 handle_command_reply: tid=0 2024-04-25T16:43:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.347+0000 7f6a46a7a740 1 client.15783 shutdown 2024-04-25T16:43:15.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.347+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 msgr2=0x5590129e56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.347+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 0x5590129e56d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009850 tx=0x7f6a30031ba0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.347+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 shutdown_connections 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5590126331b0 0x559012a5a2c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5590126124d0 0x5590129e30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55901260a2f0 0x5590129e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a1d7fa640 0 client.15783 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 --2- 172.21.15.112:0/3246462139 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5590125f1d80 0x5590129e56d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 >> 172.21.15.112:0/3246462139 conn(0x559012627300 msgr2=0x5590129d4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 shutdown_connections 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 1 -- 172.21.15.112:0/3246462139 wait complete. 2024-04-25T16:43:15.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a46a7a740 20 client.15783 trim_cache size 0 max 16384 2024-04-25T16:43:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:43:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:43:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:43:15.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6a200afc50 msgr2=0x7f6a200b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6a200afc50 0x7f6a200b2160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009fd0 tx=0x7f6a28004a40 comp rx=0 tx=0).stop 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 msgr2=0x7f6a381a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a381a1310 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00d770 tx=0x7f6a2c00dc30 comp rx=0 tx=0).stop 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 shutdown_connections 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6a200afc50 0x7f6a200b2160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38108e60 0x7f6a381a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6a38108a50 0x7f6a381a1310 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 --2- 172.21.15.112:0/945324829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a381072f0 0x7f6a381a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 >> 172.21.15.112:0/945324829 conn(0x7f6a380778b0 msgr2=0x7f6a381052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 shutdown_connections 2024-04-25T16:43:15.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:15.348+0000 7f6a3fbc5640 1 -- 172.21.15.112:0/945324829 wait complete. 2024-04-25T16:43:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:15 smithi133 ceph-mon[32691]: pgmap v147: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-04-25T16:43:15.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:15 smithi116 ceph-mon[32712]: pgmap v147: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-04-25T16:43:15.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:15 smithi112 ceph-mon[29122]: pgmap v147: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-04-25T16:43:15.876 INFO:tasks.workunit.client.0.smithi112.stdout: 1 470688 11.63 MB/sec execute 81 sec latency 7528.210 ms 2024-04-25T16:43:16.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 473017 11.66 MB/sec execute 82 sec latency 37.548 ms 2024-04-25T16:43:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:17 smithi133 ceph-mon[32691]: pgmap v148: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-04-25T16:43:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:17 smithi116 ceph-mon[32712]: pgmap v148: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-04-25T16:43:17.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:17 smithi112 ceph-mon[29122]: pgmap v148: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-04-25T16:43:17.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 475438 11.66 MB/sec execute 83 sec latency 18.442 ms 2024-04-25T16:43:18.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 477473 11.67 MB/sec execute 84 sec latency 15.943 ms 2024-04-25T16:43:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:19 smithi133 ceph-mon[32691]: pgmap v149: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-04-25T16:43:19.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:19 smithi116 ceph-mon[32712]: pgmap v149: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-04-25T16:43:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:19 smithi112 ceph-mon[29122]: pgmap v149: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 2.4 MiB/s wr, 247 op/s 2024-04-25T16:43:19.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 479957 11.70 MB/sec execute 85 sec latency 20.541 ms 2024-04-25T16:43:20.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 481529 11.63 MB/sec execute 86 sec latency 89.575 ms 2024-04-25T16:43:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:21 smithi133 ceph-mon[32691]: pgmap v150: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-04-25T16:43:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:21 smithi116 ceph-mon[32712]: pgmap v150: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-04-25T16:43:21.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:21 smithi112 ceph-mon[29122]: pgmap v150: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-04-25T16:43:21.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 483792 11.65 MB/sec execute 87 sec latency 25.778 ms 2024-04-25T16:43:22.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 486230 11.66 MB/sec execute 88 sec latency 14.205 ms 2024-04-25T16:43:23.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:23 smithi133 ceph-mon[32691]: pgmap v151: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.2 MiB/s wr, 226 op/s 2024-04-25T16:43:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:23 smithi116 ceph-mon[32712]: pgmap v151: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.2 MiB/s wr, 226 op/s 2024-04-25T16:43:23.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:23 smithi112 ceph-mon[29122]: pgmap v151: 97 pgs: 97 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 2.2 MiB/s wr, 226 op/s 2024-04-25T16:43:23.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 488280 11.66 MB/sec execute 89 sec latency 13.596 ms 2024-04-25T16:43:24.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 490698 11.69 MB/sec execute 90 sec latency 34.036 ms 2024-04-25T16:43:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:25 smithi133 ceph-mon[32691]: pgmap v152: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.8 MiB/s wr, 323 op/s 2024-04-25T16:43:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:25 smithi116 ceph-mon[32712]: pgmap v152: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.8 MiB/s wr, 323 op/s 2024-04-25T16:43:25.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:25 smithi112 ceph-mon[29122]: pgmap v152: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 3.8 MiB/s wr, 323 op/s 2024-04-25T16:43:25.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 492644 11.63 MB/sec execute 91 sec latency 98.245 ms 2024-04-25T16:43:26.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 494654 11.65 MB/sec execute 92 sec latency 23.065 ms 2024-04-25T16:43:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:27 smithi133 ceph-mon[32691]: pgmap v153: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-25T16:43:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:27 smithi116 ceph-mon[32712]: pgmap v153: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-25T16:43:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:27 smithi112 ceph-mon[29122]: pgmap v153: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-25T16:43:27.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 497363 11.70 MB/sec execute 93 sec latency 16.444 ms 2024-04-25T16:43:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 -- 172.21.15.112:0/3895610518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 msgr2=0x7f907810f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 --2- 172.21.15.112:0/3895610518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907810f5c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9070009ca0 tx=0x7f907002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 -- 172.21.15.112:0/3895610518 shutdown_connections 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 --2- 172.21.15.112:0/3895610518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078108f40 0x7f9078102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 --2- 172.21.15.112:0/3895610518 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907810f5c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 --2- 172.21.15.112:0/3895610518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108aa0 0x7f9078102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 -- 172.21.15.112:0/3895610518 >> 172.21.15.112:0/3895610518 conn(0x7f90780781a0 msgr2=0x7f90780785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 -- 172.21.15.112:0/3895610518 shutdown_connections 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 -- 172.21.15.112:0/3895610518 wait complete. 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 Processor -- start 2024-04-25T16:43:28.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.451+0000 7f907f4ee640 1 -- start start 2024-04-25T16:43:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907f4ee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907f4ee640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078108aa0 0x7f907819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907f4ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108f40 0x7f907819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907f4ee640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9078112c90 con 0x7f90781031f0 2024-04-25T16:43:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907f4ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9078112b10 con 0x7f9078108f40 2024-04-25T16:43:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907f4ee640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9078112e10 con 0x7f9078108aa0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907819b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41106/0 (socket says 172.21.15.112:41106) 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 -- 172.21.15.112:0/570522962 learned_addr learned my addr 172.21.15.112:0/570522962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907da8b640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108f40 0x7f907819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907ca89640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078108aa0 0x7f907819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 -- 172.21.15.112:0/570522962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078108aa0 msgr2=0x7f907819baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078108aa0 0x7f907819baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 -- 172.21.15.112:0/570522962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108f40 msgr2=0x7f907819c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108f40 0x7f907819c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 -- 172.21.15.112:0/570522962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90781a68c0 con 0x7f90781031f0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907da8b640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108f40 0x7f907819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f907d28a640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907819b5b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9c0 tx=0x7f907400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.452+0000 7f906e7fc640 1 -- 172.21.15.112:0/570522962 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907400cd80 con 0x7f90781031f0 2024-04-25T16:43:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.453+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781a6bb0 con 0x7f90781031f0 2024-04-25T16:43:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.453+0000 7f906e7fc640 1 -- 172.21.15.112:0/570522962 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90740054d0 con 0x7f90781031f0 2024-04-25T16:43:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.453+0000 7f906e7fc640 1 -- 172.21.15.112:0/570522962 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074010430 con 0x7f90781031f0 2024-04-25T16:43:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.453+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90781a7150 con 0x7f90781031f0 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.455+0000 7f906e7fc640 1 -- 172.21.15.112:0/570522962 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f9074004fd0 con 0x7f90781031f0 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.455+0000 7f908639d740 1 Processor -- start 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 -- start start 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f906e7fc640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f90640afc50 0x7f90640b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c298af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c299030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c299570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e64c1ff460 con 0x55e64bed84d0 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e64c1b9590 con 0x55e64bef91b0 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f908639d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e64c1fb420 con 0x55e64bed02f0 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c298af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.461 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907da8b640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f90640afc50 0x7f90640b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c298af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:40854/0 (socket says 172.21.15.112:40854) 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907da8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c299570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907ca89640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c299030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 -- 172.21.15.112:0/1502006498 learned_addr learned my addr 172.21.15.112:0/1502006498 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f906e7fc640 1 -- 172.21.15.112:0/570522962 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f90740df440 con 0x7f90781031f0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 -- 172.21.15.112:0/1502006498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 msgr2=0x55e64c299030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c299030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 -- 172.21.15.112:0/1502006498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 msgr2=0x55e64c299570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c299570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.456+0000 7f907d28a640 1 -- 172.21.15.112:0/1502006498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64c1c60d0 con 0x55e64bef91b0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.457+0000 7f907d28a640 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c298af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9074000c00 tx=0x7f90740ca050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dcd39afe20417d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.457+0000 7f907da8b640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f90640afc50 0x7f90640b2160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9078075eb0 tx=0x7f907002a8c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:28.462 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.457+0000 7f90637fe640 1 -- 172.21.15.112:0/1502006498 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740a5b80 con 0x55e64bef91b0 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.457+0000 7f90637fe640 1 -- 172.21.15.112:0/1502006498 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90740046c0 con 0x55e64bef91b0 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.457+0000 7f90637fe640 1 -- 172.21.15.112:0/1502006498 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90740a4a50 con 0x55e64bef91b0 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 -- 172.21.15.112:0/1502006498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 msgr2=0x55e64c298af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c298af0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9074000c00 tx=0x7f90740ca050 comp rx=0 tx=0).stop 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 -- 172.21.15.112:0/1502006498 shutdown_connections 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c299030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c299570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 --2- 172.21.15.112:0/1502006498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c298af0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 -- 172.21.15.112:0/1502006498 >> 172.21.15.112:0/1502006498 conn(0x55e64beed300 msgr2=0x55e64c2952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 -- 172.21.15.112:0/1502006498 shutdown_connections 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 -- 172.21.15.112:0/1502006498 wait complete. 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 -- start start 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.458+0000 7f908639d740 1 Processor -- start 2024-04-25T16:43:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c2a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c2a3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e64c2a5bc0 con 0x55e64bed84d0 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e64c186b60 con 0x55e64bef91b0 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e64c2a5d40 con 0x55e64bed02f0 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f908639d740 10 client.0 _ll_register_callbacks cb 0x55e64c1ab2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41130/0 (socket says 172.21.15.112:41130) 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907da8b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c2a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 -- 172.21.15.112:0/2897197872 learned_addr learned my addr 172.21.15.112:0/2897197872 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907d28a640 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c2a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 -- 172.21.15.112:0/2897197872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 msgr2=0x55e64c2a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c2a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 -- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 msgr2=0x55e64c2a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c2a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.459+0000 7f907ca89640 1 -- 172.21.15.112:0/2897197872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64bf11210 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f907ca89640 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a50b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f907806c400 tx=0x7f9068012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e89a649e41c970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f9060ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f9060ff9640 1 -- 172.21.15.112:0/2897197872 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068004050 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f9060ff9640 1 -- 172.21.15.112:0/2897197872 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f90680041f0 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e64c20d070 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f9060ff9640 1 -- 172.21.15.112:0/2897197872 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f90680043f0 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 10 client.15795 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e64bfdd620 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f9060ff9640 1 -- 172.21.15.112:0/2897197872 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e64c20d070 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f9060ff9640 1 -- 172.21.15.112:0/2897197872 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9068003a00 con 0x55e64bed84d0 2024-04-25T16:43:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 10 client.15795 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 20 client.15795 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64beb7d80 0x55e64c2b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 4 client.15795 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.460+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e64c206ee0 con 0x55e64beb7d80 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.461+0000 7f907d28a640 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64beb7d80 0x55e64c2b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.461+0000 7f9060ff9640 10 client.15795 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.461+0000 7f907d28a640 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64beb7d80 0x55e64c2b57c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d6aa47813129b6 server_cookie=4429811a71112dad in_seq=0 out_seq=0 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f9060ff9640 1 -- 172.21.15.112:0/2897197872 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e64c206ee0 con 0x55e64beb7d80 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f9060ff9640 10 client.15795 handle_command_reply: tid=0 2024-04-25T16:43:28.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 client.15795 shutdown 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 msgr2=0x55e64c2a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a50b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f907806c400 tx=0x7f9068012b60 comp rx=0 tx=0).stop 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 shutdown_connections 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bef91b0 0x55e64c2a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a50b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64beb7d80 0x55e64c2b57c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f9060ff9640 0 client.15795 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 --2- 172.21.15.112:0/2897197872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64bed02f0 0x55e64c2a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 >> 172.21.15.112:0/2897197872 conn(0x55e64beed300 msgr2=0x55e64c298920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 shutdown_connections 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 1 -- 172.21.15.112:0/2897197872 wait complete. 2024-04-25T16:43:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.462+0000 7f908639d740 20 client.15795 trim_cache size 0 max 16384 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.479+0000 7f908639d740 1 Processor -- start 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.479+0000 7f908639d740 1 -- start start 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f908639d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c29cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f908639d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2bafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f908639d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f908639d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90781b2490 con 0x55e64bed84d0 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f908639d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90781afd60 con 0x55e64bed02f0 2024-04-25T16:43:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f908639d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90781adbb0 con 0x55e64beb7d80 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2bb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41142/0 (socket says 172.21.15.112:41142) 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 -- 172.21.15.112:0/2627802922 learned_addr learned my addr 172.21.15.112:0/2627802922 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907d28a640 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c29cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907ca89640 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2bafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 -- 172.21.15.112:0/2627802922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 msgr2=0x55e64c29cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c29cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 -- 172.21.15.112:0/2627802922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 msgr2=0x55e64c2bafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2bafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 -- 172.21.15.112:0/2627802922 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e64c2a1560 con 0x55e64bed84d0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907d28a640 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c29cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f907da8b640 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2bb500 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9070033000 tx=0x7f9070042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa9d44eeebd2426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f9062ffd640 1 -- 172.21.15.112:0/2627802922 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9070045040 con 0x55e64bed84d0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f9062ffd640 1 -- 172.21.15.112:0/2627802922 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9070036070 con 0x55e64bed84d0 2024-04-25T16:43:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.480+0000 7f9062ffd640 1 -- 172.21.15.112:0/2627802922 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907002fa60 con 0x55e64bed84d0 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 -- 172.21.15.112:0/2627802922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 msgr2=0x55e64c2bb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2bb500 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9070033000 tx=0x7f9070042000 comp rx=0 tx=0).stop 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 -- 172.21.15.112:0/2627802922 shutdown_connections 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2bb500 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2bafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 --2- 172.21.15.112:0/2627802922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c29cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 -- 172.21.15.112:0/2627802922 >> 172.21.15.112:0/2627802922 conn(0x55e64beed300 msgr2=0x55e64c29c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 -- 172.21.15.112:0/2627802922 shutdown_connections 2024-04-25T16:43:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.481+0000 7f908639d740 1 -- 172.21.15.112:0/2627802922 wait complete. 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 -- start start 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 Processor -- start 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c2ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f90781ae220 con 0x55e64bed84d0 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90781af690 con 0x55e64bed02f0 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90781ae8f0 con 0x55e64beb7d80 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f908639d740 10 client.0 _ll_register_callbacks cb 0x7f9068009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f907da8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f907ca89640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f907da8b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41156/0 (socket says 172.21.15.112:41156) 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.482+0000 7f907da8b640 1 -- 172.21.15.112:0/2740283211 learned_addr learned my addr 172.21.15.112:0/2740283211 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907d28a640 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c2ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907ca89640 1 -- 172.21.15.112:0/2740283211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 msgr2=0x55e64c2ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907ca89640 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c2ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907ca89640 1 -- 172.21.15.112:0/2740283211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 msgr2=0x55e64c2a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907ca89640 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907ca89640 1 -- 172.21.15.112:0/2740283211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e64c2a90a0 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f907ca89640 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2a8620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9068003ea0 tx=0x7f9068004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dffb40d4d0d132cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f9060ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f9060ff9640 1 -- 172.21.15.112:0/2740283211 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9068029070 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f9060ff9640 1 -- 172.21.15.112:0/2740283211 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f906800fbf0 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9070036070 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f9060ff9640 1 -- 172.21.15.112:0/2740283211 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f906803b050 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f908639d740 10 client.25499 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.483+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e64c31d7d0 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f9060ff9640 1 -- 172.21.15.112:0/2740283211 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9070036070 con 0x55e64bed02f0 2024-04-25T16:43:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f9060ff9640 1 -- 172.21.15.112:0/2740283211 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9068036030 con 0x55e64bed02f0 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f908639d740 10 client.25499 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f908639d740 10 client.25499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f908639d740 20 client.25499 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f908639d740 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64bef91b0 0x55e64c31ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f908639d740 4 client.25499 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e64c1825f0 con 0x55e64bef91b0 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f907d28a640 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64bef91b0 0x55e64c31ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f9060ff9640 10 client.25499 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.484+0000 7f907d28a640 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64bef91b0 0x55e64c31ff50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fc45f48ab864857 server_cookie=58ce075d99545b37 in_seq=0 out_seq=0 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f9060ff9640 1 -- 172.21.15.112:0/2740283211 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e64c1825f0 con 0x55e64bef91b0 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f9060ff9640 10 client.25499 handle_command_reply: tid=0 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 client.25499 shutdown 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 msgr2=0x55e64c2a8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2a8620 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9068003ea0 tx=0x7f9068004250 comp rx=0 tx=0).stop 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 shutdown_connections 2024-04-25T16:43:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e64bed84d0 0x55e64c2a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e64bed02f0 0x55e64c2a8620 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e64bef91b0 0x55e64c31ff50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 --2- 172.21.15.112:0/2740283211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e64beb7d80 0x55e64c2ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f9060ff9640 0 client.25499 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 >> 172.21.15.112:0/2740283211 conn(0x55e64beed300 msgr2=0x55e64c2a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 shutdown_connections 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 1 -- 172.21.15.112:0/2740283211 wait complete. 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.672+0000 7f908639d740 20 client.25499 trim_cache size 0 max 16384 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:43:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:43:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:43:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f90640afc50 msgr2=0x7f90640b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f90640afc50 0x7f90640b2160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9078075eb0 tx=0x7f907002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 msgr2=0x7f907819b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907819b5b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9c0 tx=0x7f907400ee80 comp rx=0 tx=0).stop 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 shutdown_connections 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f90640afc50 0x7f90640b2160 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9078108aa0 0x7f907819baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9078108f40 0x7f907819c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 --2- 172.21.15.112:0/570522962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f90781031f0 0x7f907819b5b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 >> 172.21.15.112:0/570522962 conn(0x7f90780781a0 msgr2=0x7f90780ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 shutdown_connections 2024-04-25T16:43:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:28.673+0000 7f907f4ee640 1 -- 172.21.15.112:0/570522962 wait complete. 2024-04-25T16:43:28.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 498810 11.65 MB/sec execute 94 sec latency 105.061 ms 2024-04-25T16:43:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:29 smithi133 ceph-mon[32691]: pgmap v154: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-25T16:43:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:29 smithi116 ceph-mon[32712]: pgmap v154: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-25T16:43:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:29 smithi112 ceph-mon[29122]: pgmap v154: 97 pgs: 97 active+clean; 53 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 2.9 MiB/s wr, 176 op/s 2024-04-25T16:43:29.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 501231 11.67 MB/sec execute 95 sec latency 28.183 ms 2024-04-25T16:43:30.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 502720 11.61 MB/sec execute 96 sec latency 74.244 ms 2024-04-25T16:43:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:31 smithi133 ceph-mon[32691]: pgmap v155: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-04-25T16:43:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:31 smithi116 ceph-mon[32712]: pgmap v155: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-04-25T16:43:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:31 smithi112 ceph-mon[29122]: pgmap v155: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 308 op/s 2024-04-25T16:43:31.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 505046 11.63 MB/sec execute 97 sec latency 26.264 ms 2024-04-25T16:43:32.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 507476 11.64 MB/sec execute 98 sec latency 20.014 ms 2024-04-25T16:43:33.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:33 smithi133 ceph-mon[32691]: pgmap v156: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-25T16:43:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:33 smithi116 ceph-mon[32712]: pgmap v156: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-25T16:43:33.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:33 smithi112 ceph-mon[29122]: pgmap v156: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 229 op/s 2024-04-25T16:43:33.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 509540 11.64 MB/sec execute 99 sec latency 19.573 ms 2024-04-25T16:43:34.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 511974 11.67 MB/sec execute 100 sec latency 33.808 ms 2024-04-25T16:43:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:35 smithi133 ceph-mon[32691]: pgmap v157: 97 pgs: 97 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 322 op/s 2024-04-25T16:43:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:35 smithi116 ceph-mon[32712]: pgmap v157: 97 pgs: 97 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 322 op/s 2024-04-25T16:43:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:35 smithi112 ceph-mon[29122]: pgmap v157: 97 pgs: 97 active+clean; 42 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 322 op/s 2024-04-25T16:43:35.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 513671 11.61 MB/sec execute 101 sec latency 89.480 ms 2024-04-25T16:43:36.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 515703 11.62 MB/sec execute 102 sec latency 20.996 ms 2024-04-25T16:43:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:37 smithi133 ceph-mon[32691]: pgmap v158: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:43:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:43:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:37 smithi116 ceph-mon[32712]: pgmap v158: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:43:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:43:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:37 smithi112 ceph-mon[29122]: pgmap v158: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:43:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:43:37.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 517731 11.59 MB/sec execute 103 sec latency 25.405 ms 2024-04-25T16:43:38.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:38 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:43:38.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:38 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:43:38.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:38 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:43:38.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:38 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:43:38.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:38 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:43:38.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:38 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:43:38.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:38 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:43:38.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:38 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:43:38.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:38 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:43:38.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 519930 11.61 MB/sec execute 104 sec latency 13.439 ms 2024-04-25T16:43:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:39 smithi133 ceph-mon[32691]: pgmap v159: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:43:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:39 smithi116 ceph-mon[32712]: pgmap v159: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:43:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:39 smithi112 ceph-mon[29122]: pgmap v159: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:43:39.877 INFO:tasks.workunit.client.0.smithi112.stdout: 1 522450 11.65 MB/sec execute 105 sec latency 22.391 ms 2024-04-25T16:43:40.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 523990 11.60 MB/sec execute 106 sec latency 70.455 ms 2024-04-25T16:43:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:41 smithi133 ceph-mon[32691]: pgmap v160: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 414 op/s 2024-04-25T16:43:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:41 smithi116 ceph-mon[32712]: pgmap v160: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 414 op/s 2024-04-25T16:43:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:41 smithi112 ceph-mon[29122]: pgmap v160: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 414 op/s 2024-04-25T16:43:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 -- 172.21.15.112:0/4000914422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108f20 msgr2=0x7febb8102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 --2- 172.21.15.112:0/4000914422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108f20 0x7febb8102b60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feba0009900 tx=0x7feba002a410 comp rx=0 tx=0).stop 2024-04-25T16:43:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 -- 172.21.15.112:0/4000914422 shutdown_connections 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 --2- 172.21.15.112:0/4000914422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb81031d0 0x7febb810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 --2- 172.21.15.112:0/4000914422 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108f20 0x7febb8102b60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 --2- 172.21.15.112:0/4000914422 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb8108a80 0x7febb8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 -- 172.21.15.112:0/4000914422 >> 172.21.15.112:0/4000914422 conn(0x7febb8079170 msgr2=0x7febb80795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 -- 172.21.15.112:0/4000914422 shutdown_connections 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 -- 172.21.15.112:0/4000914422 wait complete. 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.771+0000 7febbdcb6640 1 Processor -- start 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 -- start start 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 0x7febb819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108a80 0x7febb819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb8108f20 0x7febb819c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febb8112c70 con 0x7febb8108a80 2024-04-25T16:43:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7febb8112af0 con 0x7febb81031d0 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febbdcb6640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7febb8112df0 con 0x7febb8108f20 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febaeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108a80 0x7febb819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 0x7febb819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb7fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb8108f20 0x7febb819c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 0x7febb819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:42562/0 (socket says 172.21.15.112:42562) 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febaeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108a80 0x7febb819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58964/0 (socket says 172.21.15.112:58964) 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb77fe640 1 -- 172.21.15.112:0/1906200659 learned_addr learned my addr 172.21.15.112:0/1906200659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb7fff640 1 -- 172.21.15.112:0/1906200659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 msgr2=0x7febb819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb7fff640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 0x7febb819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb7fff640 1 -- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108a80 msgr2=0x7febb819bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb7fff640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108a80 0x7febb819bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb7fff640 1 -- 172.21.15.112:0/1906200659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb81a6920 con 0x7febb8108f20 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.772+0000 7febb77fe640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 0x7febb819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.773+0000 7febb7fff640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb8108f20 0x7febb819c090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febb000e970 tx=0x7febb000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.773+0000 7febb57fa640 1 -- 172.21.15.112:0/1906200659 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb000ccc0 con 0x7febb8108f20 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.773+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb81a6c10 con 0x7febb8108f20 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.773+0000 7febb57fa640 1 -- 172.21.15.112:0/1906200659 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb0005020 con 0x7febb8108f20 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.773+0000 7febb57fa640 1 -- 172.21.15.112:0/1906200659 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0004a00 con 0x7febb8108f20 2024-04-25T16:43:41.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.773+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb81a7130 con 0x7febb8108f20 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.775+0000 7febb57fa640 1 -- 172.21.15.112:0/1906200659 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7febb0004ba0 con 0x7febb8108f20 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.775+0000 7febc4b6d740 1 Processor -- start 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.775+0000 7febc4b6d740 1 -- start start 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1809af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14412f0 0x5644b180a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14494d0 0x5644b180a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febc4b6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5644b1770460 con 0x5644b14412f0 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febc4b6d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5644b172a590 con 0x5644b146a1b0 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febc4b6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5644b176c420 con 0x5644b14494d0 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14412f0 0x5644b180a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1809af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1809af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:42568/0 (socket says 172.21.15.112:42568) 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 -- 172.21.15.112:0/1339120631 learned_addr learned my addr 172.21.15.112:0/1339120631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14494d0 0x5644b180a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febb57fa640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feba40afc50 0x7feba40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 -- 172.21.15.112:0/1339120631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14494d0 msgr2=0x5644b180a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14494d0 0x5644b180a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 -- 172.21.15.112:0/1339120631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14412f0 msgr2=0x5644b180a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14412f0 0x5644b180a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 -- 172.21.15.112:0/1339120631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b17370d0 con 0x5644b146a1b0 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febb7fff640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feba40afc50 0x7feba40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febb57fa640 1 -- 172.21.15.112:0/1906200659 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7febb00defe0 con 0x7febb8108f20 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febaeffd640 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1809af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feba00057c0 tx=0x7feba0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b8277d337083b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.776+0000 7febadffb640 1 -- 172.21.15.112:0/1339120631 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0035070 con 0x5644b146a1b0 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febadffb640 1 -- 172.21.15.112:0/1339120631 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba0004030 con 0x5644b146a1b0 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febadffb640 1 -- 172.21.15.112:0/1339120631 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba0005350 con 0x5644b146a1b0 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 -- 172.21.15.112:0/1339120631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 msgr2=0x5644b1809af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febb7fff640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feba40afc50 0x7feba40b2160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febb00a70b0 tx=0x7febb0074980 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1809af0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7feba00057c0 tx=0x7feba0000bc0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 -- 172.21.15.112:0/1339120631 shutdown_connections 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14494d0 0x5644b180a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14412f0 0x5644b180a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 --2- 172.21.15.112:0/1339120631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1809af0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 -- 172.21.15.112:0/1339120631 >> 172.21.15.112:0/1339120631 conn(0x5644b145e300 msgr2=0x5644b18062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 -- 172.21.15.112:0/1339120631 shutdown_connections 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.777+0000 7febc4b6d740 1 -- 172.21.15.112:0/1339120631 wait complete. 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 -- start start 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 Processor -- start 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b181e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 0x5644b181ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b146a1b0 0x5644b1818ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5644b1816bc0 con 0x5644b146a1b0 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5644b16f7b60 con 0x5644b14494d0 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5644b1816d40 con 0x5644b14412f0 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febc4b6d740 10 client.0 _ll_register_callbacks cb 0x5644b171c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.778+0000 7febb77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 0x5644b181ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febaeffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b146a1b0 0x5644b1818ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 0x5644b181ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:42570/0 (socket says 172.21.15.112:42570) 2024-04-25T16:43:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 -- 172.21.15.112:0/2242667224 learned_addr learned my addr 172.21.15.112:0/2242667224 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febaeffd640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b181e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 -- 172.21.15.112:0/2242667224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 msgr2=0x5644b181e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b181e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 -- 172.21.15.112:0/2242667224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b146a1b0 msgr2=0x5644b1818ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b146a1b0 0x5644b1818ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 -- 172.21.15.112:0/2242667224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b1482210 con 0x5644b14494d0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febaeffd640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b146a1b0 0x5644b1818ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febb77fe640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 0x5644b181ec10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feba800efd0 tx=0x7feba8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90da4e8e154fbf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7feb96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7feb96ffd640 1 -- 172.21.15.112:0/2242667224 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba801e060 con 0x5644b14494d0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba0004030 con 0x5644b14494d0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7feb96ffd640 1 -- 172.21.15.112:0/2242667224 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7feba8004b20 con 0x5644b14494d0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7feb96ffd640 1 -- 172.21.15.112:0/2242667224 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feba8004d20 con 0x5644b14494d0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febc4b6d740 10 client.25511 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.779+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5644b154e620 con 0x5644b14494d0 2024-04-25T16:43:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7feb96ffd640 1 -- 172.21.15.112:0/2242667224 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba0004030 con 0x5644b14494d0 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7feb96ffd640 1 -- 172.21.15.112:0/2242667224 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7feba8019070 con 0x5644b14494d0 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febc4b6d740 10 client.25511 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febc4b6d740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febc4b6d740 20 client.25511 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febc4b6d740 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b1428d80 0x5644b182aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febc4b6d740 4 client.25511 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644b177e070 con 0x5644b1428d80 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febaeffd640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b1428d80 0x5644b182aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7feb96ffd640 10 client.25511 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.780+0000 7febaeffd640 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b1428d80 0x5644b182aba0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01134d13302f934 server_cookie=1bf8351218e82d31 in_seq=0 out_seq=0 2024-04-25T16:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.784+0000 7feb96ffd640 1 -- 172.21.15.112:0/2242667224 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5644b177e070 con 0x5644b1428d80 2024-04-25T16:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.784+0000 7feb96ffd640 10 client.25511 handle_command_reply: tid=0 2024-04-25T16:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.784+0000 7febc4b6d740 1 client.25511 shutdown 2024-04-25T16:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.784+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 msgr2=0x5644b181ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.784+0000 7febc4b6d740 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 0x5644b181ec10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feba800efd0 tx=0x7feba8011810 comp rx=0 tx=0).stop 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 shutdown_connections 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b146a1b0 0x5644b1818ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14494d0 0x5644b181ec10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b1428d80 0x5644b182aba0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 --2- 172.21.15.112:0/2242667224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b181e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7feb96ffd640 0 client.25511 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 >> 172.21.15.112:0/2242667224 conn(0x5644b145e300 msgr2=0x5644b1809920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 shutdown_connections 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 1 -- 172.21.15.112:0/2242667224 wait complete. 2024-04-25T16:43:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.785+0000 7febc4b6d740 20 client.25511 trim_cache size 0 max 16384 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 Processor -- start 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 -- start start 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14412f0 0x5644b1806370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1811cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b146a1b0 0x5644b1812230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febb81b0130 con 0x5644b14494d0 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ada00 con 0x5644b14412f0 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febc4b6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ab850 con 0x5644b146a1b0 2024-04-25T16:43:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1811cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1811cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58982/0 (socket says 172.21.15.112:58982) 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febaeffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14412f0 0x5644b1806370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febb77fe640 1 -- 172.21.15.112:0/3840315475 learned_addr learned my addr 172.21.15.112:0/3840315475 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.800+0000 7febaeffd640 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b146a1b0 0x5644b1812230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febb77fe640 1 -- 172.21.15.112:0/3840315475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b146a1b0 msgr2=0x5644b1812230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febb77fe640 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b146a1b0 0x5644b1812230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febb77fe640 1 -- 172.21.15.112:0/3840315475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14412f0 msgr2=0x5644b1806370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febb77fe640 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14412f0 0x5644b1806370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febb77fe640 1 -- 172.21.15.112:0/3840315475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b1813fc0 con 0x5644b14494d0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febb77fe640 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1811cf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feba800fdb0 tx=0x7feba8004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b295cba84686dc95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febad7fa640 1 -- 172.21.15.112:0/3840315475 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8003970 con 0x5644b14494d0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febad7fa640 1 -- 172.21.15.112:0/3840315475 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba800f690 con 0x5644b14494d0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.801+0000 7febad7fa640 1 -- 172.21.15.112:0/3840315475 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba8004410 con 0x5644b14494d0 2024-04-25T16:43:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 -- 172.21.15.112:0/3840315475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 msgr2=0x5644b1811cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1811cf0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feba800fdb0 tx=0x7feba8004c10 comp rx=0 tx=0).stop 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 -- 172.21.15.112:0/3840315475 shutdown_connections 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b146a1b0 0x5644b1812230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1811cf0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3840315475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b14412f0 0x5644b1806370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 -- 172.21.15.112:0/3840315475 >> 172.21.15.112:0/3840315475 conn(0x5644b145e300 msgr2=0x5644b1805c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 -- 172.21.15.112:0/3840315475 shutdown_connections 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 -- 172.21.15.112:0/3840315475 wait complete. 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 -- start start 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.802+0000 7febc4b6d740 1 Processor -- start 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b1824c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1820030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1820570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7febb81abec0 con 0x5644b14494d0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ad330 con 0x5644b146a1b0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7febb81ac590 con 0x5644b14412f0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 10 client.0 _ll_register_callbacks cb 0x7feba8009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1820030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1820030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58984/0 (socket says 172.21.15.112:58984) 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 -- 172.21.15.112:0/3321110958 learned_addr learned my addr 172.21.15.112:0/3321110958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febaeffd640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1820570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febaeffd640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b1824c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 -- 172.21.15.112:0/3321110958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 msgr2=0x5644b1824c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b1824c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 -- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 msgr2=0x5644b1820570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1820570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 -- 172.21.15.112:0/3321110958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b1820ab0 con 0x5644b14494d0 2024-04-25T16:43:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febaeffd640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1820570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febb77fe640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1820030 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feba8005d70 tx=0x7feba800ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9d588ea7d03c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7feb96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7feb96ffd640 1 -- 172.21.15.112:0/3321110958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba800fa50 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7feb96ffd640 1 -- 172.21.15.112:0/3321110958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7feba8034480 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.803+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba800f690 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7feb96ffd640 1 -- 172.21.15.112:0/3321110958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feba8042050 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 10 client.15819 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5644b1892b80 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7feb96ffd640 1 -- 172.21.15.112:0/3321110958 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba800f690 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7feb96ffd640 1 -- 172.21.15.112:0/3321110958 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7feba8019070 con 0x5644b14494d0 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 10 client.15819 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 20 client.15819 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b13fe1c0 0x5644b1895220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 4 client.15819 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:43:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5644b1777ee0 con 0x5644b13fe1c0 2024-04-25T16:43:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.804+0000 7febaeffd640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b13fe1c0 0x5644b1895220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.805+0000 7feb96ffd640 10 client.15819 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:41.809 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.805+0000 7febaeffd640 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b13fe1c0 0x5644b1895220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f3b351a5354098 server_cookie=9ec50b8539dd137e in_seq=0 out_seq=0 2024-04-25T16:43:41.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 526196 11.62 MB/sec execute 107 sec latency 29.536 ms 2024-04-25T16:43:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7feb96ffd640 1 -- 172.21.15.112:0/3321110958 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5644b1777ee0 con 0x5644b13fe1c0 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7feb96ffd640 10 client.15819 handle_command_reply: tid=0 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 client.15819 shutdown 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 msgr2=0x5644b1820030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1820030 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7feba8005d70 tx=0x7feba800ffa0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 shutdown_connections 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644b146a1b0 0x5644b1820570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5644b14494d0 0x5644b1820030 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5644b13fe1c0 0x5644b1895220 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 --2- 172.21.15.112:0/3321110958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5644b14412f0 0x5644b1824c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7feb96ffd640 0 client.15819 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 >> 172.21.15.112:0/3321110958 conn(0x5644b145e300 msgr2=0x5644b1818350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 shutdown_connections 2024-04-25T16:43:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.951+0000 7febc4b6d740 1 -- 172.21.15.112:0/3321110958 wait complete. 2024-04-25T16:43:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.952+0000 7febc4b6d740 20 client.15819 trim_cache size 0 max 16384 2024-04-25T16:43:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:43:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:43:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:43:41.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.952+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feba40afc50 msgr2=0x7feba40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.952+0000 7febbdcb6640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feba40afc50 0x7feba40b2160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febb00a70b0 tx=0x7febb0074980 comp rx=0 tx=0).stop 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.952+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb8108f20 msgr2=0x7febb819c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.952+0000 7febbdcb6640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb8108f20 0x7febb819c090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7febb000e970 tx=0x7febb000ee30 comp rx=0 tx=0).stop 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 shutdown_connections 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feba40afc50 0x7feba40b2160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb8108f20 0x7febb819c090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febb8108a80 0x7febb819bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 --2- 172.21.15.112:0/1906200659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7febb81031d0 0x7febb819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 >> 172.21.15.112:0/1906200659 conn(0x7febb8079170 msgr2=0x7febb8101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 shutdown_connections 2024-04-25T16:43:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:41.953+0000 7febbdcb6640 1 -- 172.21.15.112:0/1906200659 wait complete. 2024-04-25T16:43:42.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 528166 11.57 MB/sec execute 108 sec latency 75.883 ms 2024-04-25T16:43:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:43 smithi133 ceph-mon[32691]: pgmap v161: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2024-04-25T16:43:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:43 smithi116 ceph-mon[32712]: pgmap v161: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2024-04-25T16:43:43.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:43 smithi112 ceph-mon[29122]: pgmap v161: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2024-04-25T16:43:43.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 530181 11.59 MB/sec execute 109 sec latency 13.640 ms 2024-04-25T16:43:44.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 532608 11.61 MB/sec execute 110 sec latency 18.492 ms 2024-04-25T16:43:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:45 smithi133 ceph-mon[32691]: pgmap v162: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 398 op/s 2024-04-25T16:43:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:45 smithi116 ceph-mon[32712]: pgmap v162: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 398 op/s 2024-04-25T16:43:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:45 smithi112 ceph-mon[29122]: pgmap v162: 97 pgs: 97 active+clean; 69 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 398 op/s 2024-04-25T16:43:45.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 534380 11.59 MB/sec execute 111 sec latency 14.686 ms 2024-04-25T16:43:46.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 536734 11.61 MB/sec execute 112 sec latency 20.393 ms 2024-04-25T16:43:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:47 smithi133 ceph-mon[32691]: pgmap v163: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 385 op/s 2024-04-25T16:43:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:47 smithi116 ceph-mon[32712]: pgmap v163: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 385 op/s 2024-04-25T16:43:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:47 smithi112 ceph-mon[29122]: pgmap v163: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.1 MiB/s wr, 385 op/s 2024-04-25T16:43:47.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 538522 11.56 MB/sec execute 113 sec latency 32.791 ms 2024-04-25T16:43:48.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 540559 11.58 MB/sec execute 114 sec latency 22.348 ms 2024-04-25T16:43:49.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:49 smithi133 ceph-mon[32691]: pgmap v164: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-04-25T16:43:49.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:49 smithi116 ceph-mon[32712]: pgmap v164: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-04-25T16:43:49.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:49 smithi112 ceph-mon[29122]: pgmap v164: 97 pgs: 97 active+clean; 64 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.3 MiB/s wr, 289 op/s 2024-04-25T16:43:49.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 542757 11.57 MB/sec execute 115 sec latency 19.515 ms 2024-04-25T16:43:50.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 544447 11.55 MB/sec execute 116 sec latency 39.274 ms 2024-04-25T16:43:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:51 smithi133 ceph-mon[32691]: pgmap v165: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 359 op/s 2024-04-25T16:43:51.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:51 smithi116 ceph-mon[32712]: pgmap v165: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 359 op/s 2024-04-25T16:43:51.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:51 smithi112 ceph-mon[29122]: pgmap v165: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 359 op/s 2024-04-25T16:43:51.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 546884 11.57 MB/sec execute 117 sec latency 57.318 ms 2024-04-25T16:43:52.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 548600 11.55 MB/sec execute 118 sec latency 16.334 ms 2024-04-25T16:43:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:53 smithi133 ceph-mon[32691]: pgmap v166: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-04-25T16:43:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:53 smithi116 ceph-mon[32712]: pgmap v166: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-04-25T16:43:53.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:53 smithi112 ceph-mon[29122]: pgmap v166: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 265 op/s 2024-04-25T16:43:53.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 551003 11.57 MB/sec execute 119 sec latency 22.203 ms 2024-04-25T16:43:54.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 552984 11.53 MB/sec execute 120 sec latency 19.016 ms 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 -- 172.21.15.112:0/2721760118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108f40 msgr2=0x7f208c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 --2- 172.21.15.112:0/2721760118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108f40 0x7f208c102b80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f207c009900 tx=0x7f207c02a410 comp rx=0 tx=0).stop 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 -- 172.21.15.112:0/2721760118 shutdown_connections 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 --2- 172.21.15.112:0/2721760118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f208c1031f0 0x7f208c10f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 --2- 172.21.15.112:0/2721760118 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108f40 0x7f208c102b80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 --2- 172.21.15.112:0/2721760118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f208c108aa0 0x7f208c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 -- 172.21.15.112:0/2721760118 >> 172.21.15.112:0/2721760118 conn(0x7f208c0781a0 msgr2=0x7f208c0785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 -- 172.21.15.112:0/2721760118 shutdown_connections 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.056+0000 7f20946d9640 1 -- 172.21.15.112:0/2721760118 wait complete. 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 Processor -- start 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 -- start start 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f208c1031f0 0x7f208c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 0x7f208c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f208c108f40 0x7f208c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f208c112c90 con 0x7f208c108aa0 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f208c112b10 con 0x7f208c1031f0 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.057+0000 7f20946d9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f208c112e10 con 0x7f208c108f40 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 0x7f208c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 0x7f208c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50502/0 (socket says 172.21.15.112:50502) 2024-04-25T16:43:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 -- 172.21.15.112:0/3408750440 learned_addr learned my addr 172.21.15.112:0/3408750440 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2092475640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f208c1031f0 0x7f208c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2092c76640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f208c108f40 0x7f208c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 -- 172.21.15.112:0/3408750440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f208c108f40 msgr2=0x7f208c19def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f208c108f40 0x7f208c19def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 -- 172.21.15.112:0/3408750440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f208c1031f0 msgr2=0x7f208c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f208c1031f0 0x7f208c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 -- 172.21.15.112:0/3408750440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208c19e530 con 0x7f208c108aa0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f2091c74640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 0x7f208c199c30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f207c02a930 tx=0x7f207c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f20837fe640 1 -- 172.21.15.112:0/3408750440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c03f030 con 0x7f208c108aa0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f20837fe640 1 -- 172.21.15.112:0/3408750440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207c004490 con 0x7f208c108aa0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208c19e7c0 con 0x7f208c108aa0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f20837fe640 1 -- 172.21.15.112:0/3408750440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c035070 con 0x7f208c108aa0 2024-04-25T16:43:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.058+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f208c19ecb0 con 0x7f208c108aa0 2024-04-25T16:43:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 Processor -- start 2024-04-25T16:43:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 -- start start 2024-04-25T16:43:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f20837fe640 1 -- 172.21.15.112:0/3408750440 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f207c042030 con 0x7f208c108aa0 2024-04-25T16:43:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 0x557a9edb0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edae940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9ee4d0 0x557a9edaeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a9ed15460 con 0x557a9e9e62f0 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557a9eccf590 con 0x557a9e9ee4d0 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.061+0000 7f209b582740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557a9ed11420 con 0x557a9ea0f1b0 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 0x557a9edb0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092475640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2091c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9ee4d0 0x557a9edaeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 0x557a9edb0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:42564/0 (socket says 172.21.15.112:42564) 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 -- 172.21.15.112:0/132689610 learned_addr learned my addr 172.21.15.112:0/132689610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 -- 172.21.15.112:0/132689610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9ee4d0 msgr2=0x557a9edaeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9ee4d0 0x557a9edaeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 -- 172.21.15.112:0/132689610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 msgr2=0x557a9edae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 -- 172.21.15.112:0/132689610 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9ecdc0d0 con 0x557a9ea0f1b0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f20837fe640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f20780afc50 0x7f20780b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092c76640 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 0x557a9edb0120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2084009fd0 tx=0x7f20840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ef058274ed287f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2092475640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f20780afc50 0x7f20780b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f20837fe640 1 -- 172.21.15.112:0/3408750440 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f207c0ff360 con 0x7f208c108aa0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2080ff9640 1 -- 172.21.15.112:0/132689610 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084004d20 con 0x557a9ea0f1b0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2080ff9640 1 -- 172.21.15.112:0/132689610 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2084004050 con 0x557a9ea0f1b0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.062+0000 7f2080ff9640 1 -- 172.21.15.112:0/132689610 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084005280 con 0x557a9ea0f1b0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 -- 172.21.15.112:0/132689610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 msgr2=0x557a9edb0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f2092475640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f20780afc50 0x7f20780b2160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x557a9edac5e0 tx=0x7f2088005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 0x557a9edb0120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2084009fd0 tx=0x7f20840118d0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 -- 172.21.15.112:0/132689610 shutdown_connections 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9ee4d0 0x557a9edaeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 --2- 172.21.15.112:0/132689610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9ea0f1b0 0x557a9edb0120 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 -- 172.21.15.112:0/132689610 >> 172.21.15.112:0/132689610 conn(0x557a9ea03300 msgr2=0x557a9edab2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 -- 172.21.15.112:0/132689610 shutdown_connections 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.063+0000 7f209b582740 1 -- 172.21.15.112:0/132689610 wait complete. 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 -- start start 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 Processor -- start 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9ee4d0 0x557a9edbf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9ea0f1b0 0x557a9edbd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a9edc00a0 con 0x557a9e9e62f0 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557a9ec9cb60 con 0x557a9ea0f1b0 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557a9edc0220 con 0x557a9e9ee4d0 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f209b582740 10 client.0 _ll_register_callbacks cb 0x557a9ecc12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092475640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092475640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edbf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50522/0 (socket says 172.21.15.112:50522) 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092475640 1 -- 172.21.15.112:0/4208917802 learned_addr learned my addr 172.21.15.112:0/4208917802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092c76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9ea0f1b0 0x557a9edbd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2091c74640 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9ee4d0 0x557a9edbf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092c76640 1 -- 172.21.15.112:0/4208917802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9ee4d0 msgr2=0x557a9edbf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092c76640 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9ee4d0 0x557a9edbf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092c76640 1 -- 172.21.15.112:0/4208917802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 msgr2=0x557a9edbf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092c76640 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edbf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.064+0000 7f2092c76640 1 -- 172.21.15.112:0/4208917802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9ea27210 con 0x557a9ea0f1b0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2092c76640 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9ea0f1b0 0x557a9edbd5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f208401b040 tx=0x7f20840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=184dbb64cd5d34c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2075ffb640 1 -- 172.21.15.112:0/4208917802 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084028020 con 0x557a9ea0f1b0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2075ffb640 1 -- 172.21.15.112:0/4208917802 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f2084004d90 con 0x557a9ea0f1b0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2084004050 con 0x557a9ea0f1b0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2075ffb640 1 -- 172.21.15.112:0/4208917802 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2084033050 con 0x557a9ea0f1b0 2024-04-25T16:43:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f209b582740 10 client.25529 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557a9eaf3620 con 0x557a9ea0f1b0 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2075ffb640 1 -- 172.21.15.112:0/4208917802 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2084004050 con 0x557a9ea0f1b0 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.065+0000 7f2075ffb640 1 -- 172.21.15.112:0/4208917802 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f20840055c0 con 0x557a9ea0f1b0 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f209b582740 10 client.25529 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f209b582740 10 client.25529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f209b582740 20 client.25529 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f209b582740 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9e9cdd80 0x557a9edcfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f209b582740 4 client.25529 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a9ed23070 con 0x557a9e9cdd80 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f2092475640 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9e9cdd80 0x557a9edcfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f2075ffb640 10 client.25529 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.066+0000 7f2092475640 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9e9cdd80 0x557a9edcfb60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df02a728d27cd8 server_cookie=8fad6fe048649751 in_seq=0 out_seq=0 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f2075ffb640 1 -- 172.21.15.112:0/4208917802 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a9ed23070 con 0x557a9e9cdd80 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f2075ffb640 10 client.25529 handle_command_reply: tid=0 2024-04-25T16:43:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 client.25529 shutdown 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9ea0f1b0 msgr2=0x557a9edbd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9ea0f1b0 0x557a9edbd5b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f208401b040 tx=0x7f20840046c0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 shutdown_connections 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9e9cdd80 0x557a9edcfb60 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9ee4d0 0x557a9edbf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9ea0f1b0 0x557a9edbd5b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f2075ffb640 0 client.25529 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 --2- 172.21.15.112:0/4208917802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9e62f0 0x557a9edbf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 >> 172.21.15.112:0/4208917802 conn(0x557a9ea03300 msgr2=0x557a9eda9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 shutdown_connections 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 1 -- 172.21.15.112:0/4208917802 wait complete. 2024-04-25T16:43:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.067+0000 7f209b582740 20 client.25529 trim_cache size 0 max 16384 2024-04-25T16:43:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 Processor -- start 2024-04-25T16:43:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 -- start start 2024-04-25T16:43:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9cdd80 0x557a9edbbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9e62f0 0x557a9edbc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edb2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1b0250 con 0x557a9e9ee4d0 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1adb20 con 0x557a9e9cdd80 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f209b582740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1ab970 con 0x557a9e9e62f0 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2092c76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edb2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2092475640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9cdd80 0x557a9edbbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2092c76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edb2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50536/0 (socket says 172.21.15.112:50536) 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2092c76640 1 -- 172.21.15.112:0/3726679761 learned_addr learned my addr 172.21.15.112:0/3726679761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2091c74640 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9e62f0 0x557a9edbc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2091c74640 1 -- 172.21.15.112:0/3726679761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9cdd80 msgr2=0x557a9edbbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2091c74640 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9cdd80 0x557a9edbbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2091c74640 1 -- 172.21.15.112:0/3726679761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 msgr2=0x557a9edb2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2091c74640 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edb2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2091c74640 1 -- 172.21.15.112:0/3726679761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a9eda6a60 con 0x557a9e9e62f0 2024-04-25T16:43:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.083+0000 7f2092c76640 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edb2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:43:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.084+0000 7f2091c74640 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9e62f0 0x557a9edbc3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f207c0fca90 tx=0x7f207c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf1425752b05322 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.084+0000 7f2077fff640 1 -- 172.21.15.112:0/3726679761 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c049040 con 0x557a9e9e62f0 2024-04-25T16:43:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.084+0000 7f2077fff640 1 -- 172.21.15.112:0/3726679761 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f207c0b0070 con 0x557a9e9e62f0 2024-04-25T16:43:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.084+0000 7f2077fff640 1 -- 172.21.15.112:0/3726679761 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0cb890 con 0x557a9e9e62f0 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- 172.21.15.112:0/3726679761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9e62f0 msgr2=0x557a9edbc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9e62f0 0x557a9edbc3c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f207c0fca90 tx=0x7f207c000bc0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- 172.21.15.112:0/3726679761 shutdown_connections 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9e62f0 0x557a9edbc3c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edb2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- 172.21.15.112:0/3726679761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9cdd80 0x557a9edbbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- 172.21.15.112:0/3726679761 >> 172.21.15.112:0/3726679761 conn(0x557a9ea03300 msgr2=0x557a9edae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- 172.21.15.112:0/3726679761 shutdown_connections 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- 172.21.15.112:0/3726679761 wait complete. 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- start start 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 Processor -- start 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9cdd80 0x557a9edc2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 0x557a9edc2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edc2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.085+0000 7f209b582740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1abfe0 con 0x557a9e9ee4d0 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f209b582740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1ad450 con 0x557a9e9e62f0 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f209b582740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f208c1ac6b0 con 0x557a9e9cdd80 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f209b582740 10 client.0 _ll_register_callbacks cb 0x7f2088009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 0x557a9edc2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2092c76640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edc2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2092475640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9cdd80 0x557a9edc2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 0x557a9edc2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:47020/0 (socket says 172.21.15.112:47020) 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 -- 172.21.15.112:0/561134163 learned_addr learned my addr 172.21.15.112:0/561134163 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 -- 172.21.15.112:0/561134163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9cdd80 msgr2=0x557a9edc2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9cdd80 0x557a9edc2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 -- 172.21.15.112:0/561134163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 msgr2=0x557a9edc2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edc2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 -- 172.21.15.112:0/561134163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a9ee35f10 con 0x557a9e9e62f0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2091c74640 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 0x557a9edc2840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f207c006010 tx=0x7f207c0c7110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=361180a295e5602 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2075ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2075ffb640 1 -- 172.21.15.112:0/561134163 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c0e1060 con 0x557a9e9e62f0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a9ed1cee0 con 0x557a9e9e62f0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2075ffb640 1 -- 172.21.15.112:0/561134163 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f207c0c3820 con 0x557a9e9e62f0 2024-04-25T16:43:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f2075ffb640 1 -- 172.21.15.112:0/561134163 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f207c049040 con 0x557a9e9e62f0 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f209b582740 10 client.25535 fetch_fsmap learned FSMap version 23 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.086+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557a9ee37180 con 0x557a9e9e62f0 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f2075ffb640 1 -- 172.21.15.112:0/561134163 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557a9ed1cee0 con 0x557a9e9e62f0 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f2075ffb640 1 -- 172.21.15.112:0/561134163 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f207c03ba30 con 0x557a9e9e62f0 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f209b582740 10 client.25535 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f209b582740 10 client.25535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f209b582740 20 client.25535 populate_metadata read hostname 'smithi112' 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f209b582740 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9ea0f1b0 0x557a9ee398f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f209b582740 4 client.25535 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a9ec985f0 con 0x557a9ea0f1b0 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f2092475640 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9ea0f1b0 0x557a9ee398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f2075ffb640 10 client.25535 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.087+0000 7f2092475640 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9ea0f1b0 0x557a9ee398f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9099bb02f724f7bc server_cookie=f9bd48230ca4956c in_seq=0 out_seq=0 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f2075ffb640 1 -- 172.21.15.112:0/561134163 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a9ec985f0 con 0x557a9ea0f1b0 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f2075ffb640 10 client.25535 handle_command_reply: tid=0 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 client.25535 shutdown 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 msgr2=0x557a9edc2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 0x557a9edc2840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f207c006010 tx=0x7f207c0c7110 comp rx=0 tx=0).stop 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 shutdown_connections 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a9e9ee4d0 0x557a9edc2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a9e9e62f0 0x557a9edc2840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a9ea0f1b0 0x557a9ee398f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 --2- 172.21.15.112:0/561134163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a9e9cdd80 0x557a9edc2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f2075ffb640 0 client.25535 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 >> 172.21.15.112:0/561134163 conn(0x557a9ea03300 msgr2=0x557a9edae880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:55.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 shutdown_connections 2024-04-25T16:43:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 1 -- 172.21.15.112:0/561134163 wait complete. 2024-04-25T16:43:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.232+0000 7f209b582740 20 client.25535 trim_cache size 0 max 16384 2024-04-25T16:43:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:43:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:43:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:43:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f20780afc50 msgr2=0x7f20780b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f20780afc50 0x7f20780b2160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x557a9edac5e0 tx=0x7f2088005f90 comp rx=0 tx=0).stop 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 msgr2=0x7f208c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 0x7f208c199c30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f207c02a930 tx=0x7f207c031a20 comp rx=0 tx=0).stop 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 shutdown_connections 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f20780afc50 0x7f20780b2160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f208c108f40 0x7f208c19def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f208c108aa0 0x7f208c199c30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 --2- 172.21.15.112:0/3408750440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f208c1031f0 0x7f208c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:43:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 >> 172.21.15.112:0/3408750440 conn(0x7f208c0781a0 msgr2=0x7f208c1011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:43:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 shutdown_connections 2024-04-25T16:43:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:43:55.233+0000 7f20946d9640 1 -- 172.21.15.112:0/3408750440 wait complete. 2024-04-25T16:43:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:55 smithi133 ceph-mon[32691]: pgmap v167: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-04-25T16:43:55.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:55 smithi116 ceph-mon[32712]: pgmap v167: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-04-25T16:43:55.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:55 smithi112 ceph-mon[29122]: pgmap v167: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 345 op/s 2024-04-25T16:43:55.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 554624 11.54 MB/sec execute 121 sec latency 100.782 ms 2024-04-25T16:43:56.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 556562 11.50 MB/sec execute 122 sec latency 50.820 ms 2024-04-25T16:43:57.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:57 smithi133 ceph-mon[32691]: pgmap v168: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-25T16:43:57.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:57 smithi116 ceph-mon[32712]: pgmap v168: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-25T16:43:57.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:57 smithi112 ceph-mon[29122]: pgmap v168: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 247 op/s 2024-04-25T16:43:57.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 558618 11.51 MB/sec execute 123 sec latency 12.733 ms 2024-04-25T16:43:58.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 561246 11.55 MB/sec execute 124 sec latency 21.040 ms 2024-04-25T16:43:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:43:59 smithi133 ceph-mon[32691]: pgmap v169: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 167 op/s 2024-04-25T16:43:59.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:43:59 smithi116 ceph-mon[32712]: pgmap v169: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 167 op/s 2024-04-25T16:43:59.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:43:59 smithi112 ceph-mon[29122]: pgmap v169: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 167 op/s 2024-04-25T16:43:59.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 562848 11.52 MB/sec execute 125 sec latency 17.042 ms 2024-04-25T16:44:00.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 564815 11.52 MB/sec execute 126 sec latency 22.301 ms 2024-04-25T16:44:01.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:01 smithi133 ceph-mon[32691]: pgmap v170: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 369 op/s 2024-04-25T16:44:01.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:01 smithi116 ceph-mon[32712]: pgmap v170: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 369 op/s 2024-04-25T16:44:01.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:01 smithi112 ceph-mon[29122]: pgmap v170: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 369 op/s 2024-04-25T16:44:01.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 566179 11.48 MB/sec execute 127 sec latency 90.918 ms 2024-04-25T16:44:02.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 568601 11.50 MB/sec execute 128 sec latency 19.923 ms 2024-04-25T16:44:03.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:03 smithi133 ceph-mon[32691]: pgmap v171: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-04-25T16:44:03.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:03 smithi116 ceph-mon[32712]: pgmap v171: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-04-25T16:44:03.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:03 smithi112 ceph-mon[29122]: pgmap v171: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 299 op/s 2024-04-25T16:44:03.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 570435 11.46 MB/sec execute 129 sec latency 19.897 ms 2024-04-25T16:44:04.878 INFO:tasks.workunit.client.0.smithi112.stdout: 1 572554 11.48 MB/sec execute 130 sec latency 12.705 ms 2024-04-25T16:44:05.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:05 smithi133 ceph-mon[32691]: pgmap v172: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 377 op/s 2024-04-25T16:44:05.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:05 smithi116 ceph-mon[32712]: pgmap v172: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 377 op/s 2024-04-25T16:44:05.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:05 smithi112 ceph-mon[29122]: pgmap v172: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.9 MiB/s wr, 377 op/s 2024-04-25T16:44:05.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 574896 11.48 MB/sec execute 131 sec latency 16.989 ms 2024-04-25T16:44:06.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 576027 11.45 MB/sec execute 132 sec latency 237.834 ms 2024-04-25T16:44:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:07 smithi133 ceph-mon[32691]: pgmap v173: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-04-25T16:44:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:07 smithi116 ceph-mon[32712]: pgmap v173: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-04-25T16:44:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:07 smithi112 ceph-mon[29122]: pgmap v173: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 340 op/s 2024-04-25T16:44:07.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 577210 11.40 MB/sec execute 133 sec latency 19.497 ms 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 -- 172.21.15.112:0/4289388975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc102450 msgr2=0x7fc1cc102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/4289388975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc102450 0x7fc1cc102860 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009900 tx=0x7fc1c802a410 comp rx=0 tx=0).stop 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 -- 172.21.15.112:0/4289388975 shutdown_connections 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/4289388975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/4289388975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1cc1074a0 0x7fc1cc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/4289388975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc102450 0x7fc1cc102860 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 -- 172.21.15.112:0/4289388975 >> 172.21.15.112:0/4289388975 conn(0x7fc1cc0778b0 msgr2=0x7fc1cc077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 -- 172.21.15.112:0/4289388975 shutdown_connections 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.337+0000 7fc1d47c7640 1 -- 172.21.15.112:0/4289388975 wait complete. 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 Processor -- start 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 -- start start 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1cc102450 0x7fc1cc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1cc1074a0 0x7fc1cc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc111b30 con 0x7fc1cc1069e0 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1119b0 con 0x7fc1cc102450 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d47c7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc111cb0 con 0x7fc1cc1074a0 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36980/0 (socket says 172.21.15.112:36980) 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 -- 172.21.15.112:0/301975913 learned_addr learned my addr 172.21.15.112:0/301975913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d2563640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1cc102450 0x7fc1cc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1cc1074a0 0x7fc1cc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 -- 172.21.15.112:0/301975913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1cc1074a0 msgr2=0x7fc1cc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1cc1074a0 0x7fc1cc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 -- 172.21.15.112:0/301975913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1cc102450 msgr2=0x7fc1cc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1cc102450 0x7fc1cc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.338+0000 7fc1d1d62640 1 -- 172.21.15.112:0/301975913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1cc19d3d0 con 0x7fc1cc1069e0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.339+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc198ad0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00e9f0 tx=0x7fc1bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.339+0000 7fc1c37fe640 1 -- 172.21.15.112:0/301975913 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc00ccc0 con 0x7fc1cc1069e0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.339+0000 7fc1c37fe640 1 -- 172.21.15.112:0/301975913 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc0054d0 con 0x7fc1cc1069e0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.339+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1cc19d6c0 con 0x7fc1cc1069e0 2024-04-25T16:44:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.339+0000 7fc1c37fe640 1 -- 172.21.15.112:0/301975913 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc010430 con 0x7fc1cc1069e0 2024-04-25T16:44:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.339+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1cc1a5040 con 0x7fc1cc1069e0 2024-04-25T16:44:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.341+0000 7fc1c37fe640 1 -- 172.21.15.112:0/301975913 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fc1bc0046c0 con 0x7fc1cc1069e0 2024-04-25T16:44:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.341+0000 7fc1db673740 1 Processor -- start 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 -- start start 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc379940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb12f0 0x561efc379e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb94d0 0x561efc37a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561efc2e0460 con 0x561efbfb12f0 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561efc29a590 con 0x561efbfda1b0 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1db673740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561efc2dc420 con 0x561efbfb94d0 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1c37fe640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc1b80afcd0 0x7fc1b80b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2d64640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc379940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb12f0 0x561efc379e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2d64640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc379940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:35628/0 (socket says 172.21.15.112:35628) 2024-04-25T16:44:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2563640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb12f0 0x561efc379e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36984/0 (socket says 172.21.15.112:36984) 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2d64640 1 -- 172.21.15.112:0/1526848939 learned_addr learned my addr 172.21.15.112:0/1526848939 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc1b80afcd0 0x7fc1b80b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb94d0 0x561efc37a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1c37fe640 1 -- 172.21.15.112:0/301975913 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fc1bc0df360 con 0x7fc1cc1069e0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2d64640 1 -- 172.21.15.112:0/1526848939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb94d0 msgr2=0x561efc37a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.342+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb94d0 0x561efc37a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1d2d64640 1 -- 172.21.15.112:0/1526848939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb12f0 msgr2=0x561efc379e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb12f0 0x561efc379e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1d2d64640 1 -- 172.21.15.112:0/1526848939 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561efc2a70d0 con 0x561efbfda1b0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc379940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400ff80 tx=0x7fc1c400d870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9733d5adbb5634f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc1b80afcd0 0x7fc1b80b21e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fd0 tx=0x7fc1c4019000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1c0ff9640 1 -- 172.21.15.112:0/1526848939 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4018040 con 0x561efbfda1b0 2024-04-25T16:44:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1c0ff9640 1 -- 172.21.15.112:0/1526848939 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c400f610 con 0x561efbfda1b0 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.343+0000 7fc1c0ff9640 1 -- 172.21.15.112:0/1526848939 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4004670 con 0x561efbfda1b0 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 -- 172.21.15.112:0/1526848939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 msgr2=0x561efc379940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc379940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400ff80 tx=0x7fc1c400d870 comp rx=0 tx=0).stop 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 -- 172.21.15.112:0/1526848939 shutdown_connections 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb94d0 0x561efc37a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb12f0 0x561efc379e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 --2- 172.21.15.112:0/1526848939 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc379940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 -- 172.21.15.112:0/1526848939 >> 172.21.15.112:0/1526848939 conn(0x561efbfce300 msgr2=0x561efc3762a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 -- 172.21.15.112:0/1526848939 shutdown_connections 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 -- 172.21.15.112:0/1526848939 wait complete. 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 -- start start 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.344+0000 7fc1db673740 1 Processor -- start 2024-04-25T16:44:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfb12f0 0x561efc388320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc384680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfda1b0 0x561efc384bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561efc37cbb0 con 0x561efbfb94d0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561efc267b60 con 0x561efbfb12f0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561efc37cd30 con 0x561efbfda1b0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 10 client.0 _ll_register_callbacks cb 0x561efc28c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc384680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d2563640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfb12f0 0x561efc388320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc384680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36996/0 (socket says 172.21.15.112:36996) 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2495719170 learned_addr learned my addr 172.21.15.112:0/2495719170 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfb12f0 0x561efc388320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:35644/0 (socket says 172.21.15.112:35644) 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfda1b0 0x561efc384bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2495719170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfda1b0 msgr2=0x561efc384bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfda1b0 0x561efc384bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfb12f0 msgr2=0x561efc388320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfb12f0 0x561efc388320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2495719170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561efbff2210 con 0x561efbfb94d0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc384680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0dd090 tx=0x7fc1bc071cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6bbee60dd7a096 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2495719170 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0a6e30 con 0x561efbfb94d0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2495719170 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fc1bc005000 con 0x561efbfb94d0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.345+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c400f610 con 0x561efbfb94d0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2495719170 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc0a6c50 con 0x561efbfb94d0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 10 client.15849 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561efc0be620 con 0x561efbfb94d0 2024-04-25T16:44:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2495719170 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1c400f610 con 0x561efbfb94d0 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2495719170 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1bc0a2e50 con 0x561efbfb94d0 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 10 client.15849 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 20 client.15849 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf98d80 0x561efc39aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 4 client.15849 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561efc2ee070 con 0x561efbf98d80 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.346+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf98d80 0x561efc39aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.347+0000 7fc1b5ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.347+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf98d80 0x561efc39aad0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4b20a09f6c3d67 server_cookie=9d4590a10f20e00d in_seq=0 out_seq=0 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.347+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2495719170 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561efc2ee070 con 0x561efbf98d80 2024-04-25T16:44:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.347+0000 7fc1b5ffb640 10 client.15849 handle_command_reply: tid=0 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.347+0000 7fc1db673740 1 client.15849 shutdown 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 msgr2=0x561efc384680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc384680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0dd090 tx=0x7fc1bc071cb0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 shutdown_connections 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf98d80 0x561efc39aad0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfda1b0 0x561efc384bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc384680 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 --2- 172.21.15.112:0/2495719170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfb12f0 0x561efc388320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 >> 172.21.15.112:0/2495719170 conn(0x561efbfce300 msgr2=0x561efc37a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1b5ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 shutdown_connections 2024-04-25T16:44:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 1 -- 172.21.15.112:0/2495719170 wait complete. 2024-04-25T16:44:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.348+0000 7fc1db673740 20 client.15849 trim_cache size 0 max 16384 2024-04-25T16:44:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 Processor -- start 2024-04-25T16:44:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 -- start start 2024-04-25T16:44:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc37df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc37e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc3a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1b01c0 con 0x561efbfb94d0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1ada90 con 0x561efbfda1b0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1db673740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1ab8e0 con 0x561efbfb12f0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc37e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc37e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36998/0 (socket says 172.21.15.112:36998) 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2244035076 learned_addr learned my addr 172.21.15.112:0/2244035076 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc3a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc37df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2244035076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 msgr2=0x561efc37df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc37df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2244035076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 msgr2=0x561efc3a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc3a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.363+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2244035076 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561efc380240 con 0x561efbfb94d0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.364+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc37e490 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0a7b40 tx=0x7fc1bc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6d53b8e975b38a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:08.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.364+0000 7fc1b7fff640 1 -- 172.21.15.112:0/2244035076 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0c0020 con 0x561efbfb94d0 2024-04-25T16:44:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.364+0000 7fc1b7fff640 1 -- 172.21.15.112:0/2244035076 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc005680 con 0x561efbfb94d0 2024-04-25T16:44:08.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.364+0000 7fc1b7fff640 1 -- 172.21.15.112:0/2244035076 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0a7be0 con 0x561efbfb94d0 2024-04-25T16:44:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- 172.21.15.112:0/2244035076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 msgr2=0x561efc37e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc37e490 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0a7b40 tx=0x7fc1bc00ce60 comp rx=0 tx=0).stop 2024-04-25T16:44:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- 172.21.15.112:0/2244035076 shutdown_connections 2024-04-25T16:44:08.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc3a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc37e490 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- 172.21.15.112:0/2244035076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc37df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- 172.21.15.112:0/2244035076 >> 172.21.15.112:0/2244035076 conn(0x561efbfce300 msgr2=0x561efc382f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- 172.21.15.112:0/2244035076 shutdown_connections 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- 172.21.15.112:0/2244035076 wait complete. 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- start start 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 Processor -- start 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc3922b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc38d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc38db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1abf50 con 0x561efbfb94d0 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1ad3c0 con 0x561efbfda1b0 2024-04-25T16:44:08.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1cc1ac620 con 0x561efbfb12f0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1db673740 10 client.0 _ll_register_callbacks cb 0x7fc1bc00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc38d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1d1d62640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc38d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37008/0 (socket says 172.21.15.112:37008) 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.365+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2935946324 learned_addr learned my addr 172.21.15.112:0/2935946324 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d2d64640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc38db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc3922b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2935946324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 msgr2=0x561efc3922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc3922b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 msgr2=0x561efc38db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc38db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d1d62640 1 -- 172.21.15.112:0/2935946324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561efc38e080 con 0x561efbfb94d0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1d1d62640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc38d600 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00e9c0 tx=0x7fc1bc0052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20306d1395d12674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:08.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2935946324 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0c0070 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2935946324 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fc1bc0a6950 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1bc005680 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2935946324 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc0a6e30 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1db673740 10 client.15855 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x561efc38e080 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2935946324 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1bc005680 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.366+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2935946324 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc1bc0aca20 con 0x561efbfb94d0 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1db673740 10 client.15855 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1db673740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1db673740 20 client.15855 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1db673740 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf6e1c0 0x561efc404d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1db673740 4 client.15855 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:44:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561efc2e7ee0 con 0x561efbf6e1c0 2024-04-25T16:44:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf6e1c0 0x561efc404d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1d2563640 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf6e1c0 0x561efc404d20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bf8a8cf54445f6b server_cookie=89ef522a563b972b in_seq=0 out_seq=0 2024-04-25T16:44:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.367+0000 7fc1b5ffb640 10 client.15855 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1b5ffb640 1 -- 172.21.15.112:0/2935946324 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561efc2e7ee0 con 0x561efbf6e1c0 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1b5ffb640 10 client.15855 handle_command_reply: tid=0 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 client.15855 shutdown 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 msgr2=0x561efc38d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc38d600 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00e9c0 tx=0x7fc1bc0052b0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 shutdown_connections 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561efbfda1b0 0x561efc38db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561efbfb94d0 0x561efc38d600 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561efbf6e1c0 0x561efc404d20 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1b5ffb640 0 client.15855 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 --2- 172.21.15.112:0/2935946324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561efbfb12f0 0x561efc3922b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 >> 172.21.15.112:0/2935946324 conn(0x561efbfce300 msgr2=0x561efc382f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:08.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 shutdown_connections 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 1 -- 172.21.15.112:0/2935946324 wait complete. 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.498+0000 7fc1db673740 20 client.15855 trim_cache size 0 max 16384 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.500+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc1b80afcd0 msgr2=0x7fc1b80b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.500+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc1b80afcd0 0x7fc1b80b21e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fd0 tx=0x7fc1c4019000 comp rx=0 tx=0).stop 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.500+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 msgr2=0x7fc1cc198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:08.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.500+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc198ad0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00e9f0 tx=0x7fc1bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 shutdown_connections 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc1b80afcd0 0x7fc1b80b21e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1cc1074a0 0x7fc1cc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1cc1069e0 0x7fc1cc198ad0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 --2- 172.21.15.112:0/301975913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc1cc102450 0x7fc1cc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 >> 172.21.15.112:0/301975913 conn(0x7fc1cc0778b0 msgr2=0x7fc1cc100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 shutdown_connections 2024-04-25T16:44:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:08.502+0000 7fc1d47c7640 1 -- 172.21.15.112:0/301975913 wait complete. 2024-04-25T16:44:08.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 578960 11.40 MB/sec execute 134 sec latency 88.223 ms 2024-04-25T16:44:09.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:09 smithi116 ceph-mon[32712]: pgmap v174: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-04-25T16:44:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:09 smithi112 ceph-mon[29122]: pgmap v174: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-04-25T16:44:09.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:09 smithi133 ceph-mon[32691]: pgmap v174: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-04-25T16:44:09.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 580543 11.37 MB/sec execute 135 sec latency 15.036 ms 2024-04-25T16:44:10.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 582924 11.39 MB/sec execute 136 sec latency 17.216 ms 2024-04-25T16:44:11.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:11 smithi116 ceph-mon[32712]: pgmap v175: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 493 op/s 2024-04-25T16:44:11.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:11 smithi112 ceph-mon[29122]: pgmap v175: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 493 op/s 2024-04-25T16:44:11.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:11 smithi133 ceph-mon[32691]: pgmap v175: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 493 op/s 2024-04-25T16:44:11.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 584731 11.35 MB/sec execute 137 sec latency 63.238 ms 2024-04-25T16:44:12.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 586690 11.36 MB/sec execute 138 sec latency 16.037 ms 2024-04-25T16:44:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:13 smithi116 ceph-mon[32712]: pgmap v176: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 291 op/s 2024-04-25T16:44:13.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:13 smithi112 ceph-mon[29122]: pgmap v176: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 291 op/s 2024-04-25T16:44:13.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:13 smithi133 ceph-mon[32691]: pgmap v176: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 291 op/s 2024-04-25T16:44:13.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 589070 11.37 MB/sec execute 139 sec latency 18.998 ms 2024-04-25T16:44:14.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 591005 11.37 MB/sec execute 140 sec latency 14.213 ms 2024-04-25T16:44:15.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:15 smithi116 ceph-mon[32712]: pgmap v177: 97 pgs: 97 active+clean; 72 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-04-25T16:44:15.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:15 smithi112 ceph-mon[29122]: pgmap v177: 97 pgs: 97 active+clean; 72 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-04-25T16:44:15.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:15 smithi133 ceph-mon[32691]: pgmap v177: 97 pgs: 97 active+clean; 72 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-04-25T16:44:15.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 593486 11.39 MB/sec execute 141 sec latency 17.130 ms 2024-04-25T16:44:16.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 595079 11.35 MB/sec execute 142 sec latency 59.735 ms 2024-04-25T16:44:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:17 smithi116 ceph-mon[32712]: pgmap v178: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-04-25T16:44:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:17 smithi112 ceph-mon[29122]: pgmap v178: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-04-25T16:44:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:17 smithi133 ceph-mon[32691]: pgmap v178: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-04-25T16:44:17.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 597317 11.37 MB/sec execute 143 sec latency 17.914 ms 2024-04-25T16:44:18.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 599632 11.38 MB/sec execute 144 sec latency 16.514 ms 2024-04-25T16:44:19.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:19 smithi116 ceph-mon[32712]: pgmap v179: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:44:19.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:19 smithi112 ceph-mon[29122]: pgmap v179: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:44:19.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:19 smithi133 ceph-mon[32691]: pgmap v179: 97 pgs: 97 active+clean; 73 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:44:19.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 601410 11.36 MB/sec execute 145 sec latency 46.164 ms 2024-04-25T16:44:20.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 603949 11.39 MB/sec execute 146 sec latency 22.436 ms 2024-04-25T16:44:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:21 smithi116 ceph-mon[32712]: pgmap v180: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 370 op/s 2024-04-25T16:44:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:21 smithi112 ceph-mon[29122]: pgmap v180: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 370 op/s 2024-04-25T16:44:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 -- 172.21.15.112:0/3346277508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c104d30 msgr2=0x7f051c105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 --2- 172.21.15.112:0/3346277508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c104d30 0x7f051c105140 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f050c009860 tx=0x7f050c02a3d0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 -- 172.21.15.112:0/3346277508 shutdown_connections 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 --2- 172.21.15.112:0/3346277508 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 0x7f051c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 --2- 172.21.15.112:0/3346277508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c10e0f0 0x7f051c1105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 --2- 172.21.15.112:0/3346277508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c104d30 0x7f051c105140 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 -- 172.21.15.112:0/3346277508 >> 172.21.15.112:0/3346277508 conn(0x7f051c0fe260 msgr2=0x7f051c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 -- 172.21.15.112:0/3346277508 shutdown_connections 2024-04-25T16:44:21.603 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.604+0000 7f0520897640 1 -- 172.21.15.112:0/3346277508 wait complete. 2024-04-25T16:44:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 Processor -- start 2024-04-25T16:44:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 -- start start 2024-04-25T16:44:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c104d30 0x7f051c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 0x7f051c19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c10e0f0 0x7f051c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f051c113da0 con 0x7f051c104d30 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f051c113c20 con 0x7f051c10e0f0 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0520897640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f051c113f20 con 0x7f051c10b790 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f051a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c104d30 0x7f051c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f051a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c104d30 0x7f051c19a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58046/0 (socket says 172.21.15.112:58046) 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f051a59c640 1 -- 172.21.15.112:0/541709302 learned_addr learned my addr 172.21.15.112:0/541709302 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f0513fff640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 0x7f051c19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.605+0000 7f051ad9d640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c10e0f0 0x7f051c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0513fff640 1 -- 172.21.15.112:0/541709302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c10e0f0 msgr2=0x7f051c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0513fff640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c10e0f0 0x7f051c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0513fff640 1 -- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c104d30 msgr2=0x7f051c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0513fff640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c104d30 0x7f051c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0513fff640 1 -- 172.21.15.112:0/541709302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c19f5a0 con 0x7f051c10b790 2024-04-25T16:44:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0513fff640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 0x7f051c19aca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f050400e9f0 tx=0x7f050400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f05137fe640 1 -- 172.21.15.112:0/541709302 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400ccc0 con 0x7f051c10b790 2024-04-25T16:44:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f05137fe640 1 -- 172.21.15.112:0/541709302 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05040108e0 con 0x7f051c10b790 2024-04-25T16:44:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f05137fe640 1 -- 172.21.15.112:0/541709302 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504018880 con 0x7f051c10b790 2024-04-25T16:44:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c19f890 con 0x7f051c10b790 2024-04-25T16:44:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.606+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c1a7210 con 0x7f051c10b790 2024-04-25T16:44:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 Processor -- start 2024-04-25T16:44:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 -- start start 2024-04-25T16:44:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f05137fe640 1 -- 172.21.15.112:0/541709302 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f0504018a20 con 0x7f051c10b790 2024-04-25T16:44:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 0x5647c4fc8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fc9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c084d0 0x5647c4fc9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647c4f2f460 con 0x5647c4c084d0 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5647c4ee9590 con 0x5647c4c291b0 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.609+0000 7f0527821740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5647c4f2b420 con 0x5647c4c002f0 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 0x5647c4fc8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 0x5647c4fc8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:60326/0 (socket says 172.21.15.112:60326) 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f0513fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c084d0 0x5647c4fc9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 -- 172.21.15.112:0/752536094 learned_addr learned my addr 172.21.15.112:0/752536094 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051a59c640 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fc9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 -- 172.21.15.112:0/752536094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 msgr2=0x5647c4fc9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fc9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 -- 172.21.15.112:0/752536094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c084d0 msgr2=0x5647c4fc9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c084d0 0x5647c4fc9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 -- 172.21.15.112:0/752536094 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c4ef60d0 con 0x5647c4c291b0 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051ad9d640 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 0x5647c4fc8af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05140099f0 tx=0x7f05140095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=634d7f3edc8571ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f05137fe640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f05080afc50 0x7f05080b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f0510ff9640 1 -- 172.21.15.112:0/752536094 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514022070 con 0x5647c4c291b0 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f051a59c640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f05080afc50 0x7f05080b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f0510ff9640 1 -- 172.21.15.112:0/752536094 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0514003e40 con 0x5647c4c291b0 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f0510ff9640 1 -- 172.21.15.112:0/752536094 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051400eb00 con 0x5647c4c291b0 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.610+0000 7f05137fe640 1 -- 172.21.15.112:0/541709302 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f05040df3d0 con 0x7f051c10b790 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 -- 172.21.15.112:0/752536094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 msgr2=0x5647c4fc8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 0x5647c4fc8af0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05140099f0 tx=0x7f05140095c0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f051a59c640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f05080afc50 0x7f05080b2160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5647c4fc65e0 tx=0x7f050c0058d0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 -- 172.21.15.112:0/752536094 shutdown_connections 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fc9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c084d0 0x5647c4fc9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 --2- 172.21.15.112:0/752536094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c291b0 0x5647c4fc8af0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 -- 172.21.15.112:0/752536094 >> 172.21.15.112:0/752536094 conn(0x5647c4c1d300 msgr2=0x5647c4fc52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 -- 172.21.15.112:0/752536094 shutdown_connections 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.611+0000 7f0527821740 1 -- 172.21.15.112:0/752536094 wait complete. 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 -- start start 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 Processor -- start 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fd4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c4fd5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fd3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647c4fd5c20 con 0x5647c4c291b0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5647c4eb6b60 con 0x5647c4c084d0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5647c4fd5da0 con 0x5647c4c002f0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0527821740 10 client.0 _ll_register_callbacks cb 0x5647c4edb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fd3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fd3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58052/0 (socket says 172.21.15.112:58052) 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f051ad9d640 1 -- 172.21.15.112:0/4181993818 learned_addr learned my addr 172.21.15.112:0/4181993818 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f0513fff640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c4fd5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.612+0000 7f051a59c640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fd4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051ad9d640 1 -- 172.21.15.112:0/4181993818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 msgr2=0x5647c4fd4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051ad9d640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fd4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051ad9d640 1 -- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 msgr2=0x5647c4fd5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051ad9d640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c4fd5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051ad9d640 1 -- 172.21.15.112:0/4181993818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c4c41210 con 0x5647c4c291b0 2024-04-25T16:44:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051a59c640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fd4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f051ad9d640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fd3000 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f051400e720 tx=0x7f051400eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02d09f51920956f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0501ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0501ffb640 1 -- 172.21.15.112:0/4181993818 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514022040 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0501ffb640 1 -- 172.21.15.112:0/4181993818 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f0514004050 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5647c4f3d070 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0501ffb640 1 -- 172.21.15.112:0/4181993818 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0514036050 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0527821740 10 client.15867 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5647c4d0d620 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.613+0000 7f0501ffb640 1 -- 172.21.15.112:0/4181993818 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5647c4f3d070 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0501ffb640 1 -- 172.21.15.112:0/4181993818 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f051400b040 con 0x5647c4c291b0 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0527821740 10 client.15867 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0527821740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0527821740 20 client.15867 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0527821740 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4be7d80 0x5647c4fe5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0527821740 4 client.15867 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647c4f36ee0 con 0x5647c4be7d80 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f051a59c640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4be7d80 0x5647c4fe5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f0501ffb640 10 client.15867 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.614+0000 7f051a59c640 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4be7d80 0x5647c4fe5660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2db72eb5f402d2b server_cookie=edd32697cbccb606 in_seq=0 out_seq=0 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.615+0000 7f0501ffb640 1 -- 172.21.15.112:0/4181993818 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5647c4f36ee0 con 0x5647c4be7d80 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.615+0000 7f0501ffb640 10 client.15867 handle_command_reply: tid=0 2024-04-25T16:44:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 client.15867 shutdown 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 msgr2=0x5647c4fd3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fd3000 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f051400e720 tx=0x7f051400eab0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 shutdown_connections 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fd3000 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c4fd5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4be7d80 0x5647c4fe5660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 --2- 172.21.15.112:0/4181993818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c4fd4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0501ffb640 0 client.15867 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 >> 172.21.15.112:0/4181993818 conn(0x5647c4c1d300 msgr2=0x5647c4fc8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 shutdown_connections 2024-04-25T16:44:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 1 -- 172.21.15.112:0/4181993818 wait complete. 2024-04-25T16:44:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.616+0000 7f0527821740 20 client.15867 trim_cache size 0 max 16384 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 Processor -- start 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 -- start start 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c002f0 0x5647c4fccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c084d0 0x5647c4feac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4feb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1b0270 con 0x5647c4c291b0 2024-04-25T16:44:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1adb40 con 0x5647c4c002f0 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0527821740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1ab990 con 0x5647c4c084d0 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4feb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f051a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c002f0 0x5647c4fccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4feb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58066/0 (socket says 172.21.15.112:58066) 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f051ad9d640 1 -- 172.21.15.112:0/2100861553 learned_addr learned my addr 172.21.15.112:0/2100861553 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0513fff640 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c084d0 0x5647c4feac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0513fff640 1 -- 172.21.15.112:0/2100861553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c002f0 msgr2=0x5647c4fccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0513fff640 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c002f0 0x5647c4fccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0513fff640 1 -- 172.21.15.112:0/2100861553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 msgr2=0x5647c4feb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0513fff640 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4feb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f0513fff640 1 -- 172.21.15.112:0/2100861553 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647c4fcebc0 con 0x5647c4c084d0 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.632+0000 7f051ad9d640 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4feb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:44:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.633+0000 7f0513fff640 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c084d0 0x5647c4feac30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f05040dcb00 tx=0x7f0504009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188dd3e1ac6c402b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.633+0000 7f0503fff640 1 -- 172.21.15.112:0/2100861553 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040a6f50 con 0x5647c4c084d0 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.633+0000 7f0503fff640 1 -- 172.21.15.112:0/2100861553 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05040a7110 con 0x5647c4c084d0 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.633+0000 7f0503fff640 1 -- 172.21.15.112:0/2100861553 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040a6d70 con 0x5647c4c084d0 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 -- 172.21.15.112:0/2100861553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c084d0 msgr2=0x5647c4feac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c084d0 0x5647c4feac30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f05040dcb00 tx=0x7f0504009700 comp rx=0 tx=0).stop 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 -- 172.21.15.112:0/2100861553 shutdown_connections 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c084d0 0x5647c4feac30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4feb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 --2- 172.21.15.112:0/2100861553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c002f0 0x5647c4fccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 -- 172.21.15.112:0/2100861553 >> 172.21.15.112:0/2100861553 conn(0x5647c4c1d300 msgr2=0x5647c4fcc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 -- 172.21.15.112:0/2100861553 shutdown_connections 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 -- 172.21.15.112:0/2100861553 wait complete. 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 -- start start 2024-04-25T16:44:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 Processor -- start 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c5045d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.634+0000 7f0527821740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c5046280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0527821740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fda5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0527821740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1ac000 con 0x5647c4c291b0 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0527821740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1ad470 con 0x5647c4c084d0 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0527821740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f051c1ac6d0 con 0x5647c4c002f0 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0527821740 10 client.0 _ll_register_callbacks cb 0x7f0514020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c5045d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0513fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c5046280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c5045d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:50414/0 (socket says 172.21.15.112:50414) 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 -- 172.21.15.112:0/183974188 learned_addr learned my addr 172.21.15.112:0/183974188 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fda5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 -- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 msgr2=0x5647c5046280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c5046280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 -- 172.21.15.112:0/183974188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 msgr2=0x5647c4fda5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fda5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 -- 172.21.15.112:0/183974188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647c4fdabc0 con 0x5647c4c002f0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051ad9d640 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f051a59c640 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c5045d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f050c005eb0 tx=0x7f050c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b06cf810c28c4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.635+0000 7f0501ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0501ffb640 1 -- 172.21.15.112:0/183974188 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c056020 con 0x5647c4c002f0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05040a7110 con 0x5647c4c002f0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0501ffb640 1 -- 172.21.15.112:0/183974188 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f050c004040 con 0x5647c4c002f0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0501ffb640 1 -- 172.21.15.112:0/183974188 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f050c004240 con 0x5647c4c002f0 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 10 client.25507 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5647c504d350 con 0x5647c4c002f0 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0501ffb640 1 -- 172.21.15.112:0/183974188 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05040a7110 con 0x5647c4c002f0 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0501ffb640 1 -- 172.21.15.112:0/183974188 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f050c0049d0 con 0x5647c4c002f0 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 10 client.25507 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 10 client.25507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 20 client.25507 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4bbd1c0 0x5647c504fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 4 client.25507 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647c4eb25f0 con 0x5647c4bbd1c0 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.636+0000 7f0513fff640 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4bbd1c0 0x5647c504fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.637+0000 7f0501ffb640 10 client.25507 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:21.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.637+0000 7f0513fff640 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4bbd1c0 0x5647c504fb40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ebc0e5cdde3e55 server_cookie=6eae3ec6a87f72a7 in_seq=0 out_seq=0 2024-04-25T16:44:21.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:21 smithi133 ceph-mon[32691]: pgmap v180: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 370 op/s 2024-04-25T16:44:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0501ffb640 1 -- 172.21.15.112:0/183974188 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647c4eb25f0 con 0x5647c4bbd1c0 2024-04-25T16:44:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0501ffb640 10 client.25507 handle_command_reply: tid=0 2024-04-25T16:44:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 client.25507 shutdown 2024-04-25T16:44:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 msgr2=0x5647c5045d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c5045d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f050c005eb0 tx=0x7f050c005f90 comp rx=0 tx=0).stop 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 shutdown_connections 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5647c4c291b0 0x5647c4fda5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5647c4c084d0 0x5647c5046280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5647c4bbd1c0 0x5647c504fb40 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 --2- 172.21.15.112:0/183974188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5647c4c002f0 0x5647c5045d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0501ffb640 0 client.25507 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.784+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 >> 172.21.15.112:0/183974188 conn(0x5647c4c1d300 msgr2=0x5647c4fc8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.785+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 shutdown_connections 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.785+0000 7f0527821740 1 -- 172.21.15.112:0/183974188 wait complete. 2024-04-25T16:44:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.785+0000 7f0527821740 20 client.25507 trim_cache size 0 max 16384 2024-04-25T16:44:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:44:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:44:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:44:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f05080afc50 msgr2=0x7f05080b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f05080afc50 0x7f05080b2160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5647c4fc65e0 tx=0x7f050c0058d0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 msgr2=0x7f051c19aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 0x7f051c19aca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f050400e9f0 tx=0x7f050400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 shutdown_connections 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f05080afc50 0x7f05080b2160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f051c10b790 0x7f051c19aca0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f051c10e0f0 0x7f051c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 --2- 172.21.15.112:0/541709302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f051c104d30 0x7f051c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 >> 172.21.15.112:0/541709302 conn(0x7f051c0fe260 msgr2=0x7f051c10e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 shutdown_connections 2024-04-25T16:44:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:21.786+0000 7f0520897640 1 -- 172.21.15.112:0/541709302 wait complete. 2024-04-25T16:44:21.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 605213 11.35 MB/sec execute 147 sec latency 71.410 ms 2024-04-25T16:44:22.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 607485 11.36 MB/sec execute 148 sec latency 36.722 ms 2024-04-25T16:44:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:23 smithi116 ceph-mon[32712]: pgmap v181: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-04-25T16:44:23.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:23 smithi112 ceph-mon[29122]: pgmap v181: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-04-25T16:44:23.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:23 smithi133 ceph-mon[32691]: pgmap v181: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 199 op/s 2024-04-25T16:44:23.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 609048 11.33 MB/sec execute 149 sec latency 15.338 ms 2024-04-25T16:44:24.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 611437 11.35 MB/sec execute 150 sec latency 16.229 ms 2024-04-25T16:44:25.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:25 smithi116 ceph-mon[32712]: pgmap v182: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 335 op/s 2024-04-25T16:44:25.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:25 smithi112 ceph-mon[29122]: pgmap v182: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 335 op/s 2024-04-25T16:44:25.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:25 smithi133 ceph-mon[32691]: pgmap v182: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 335 op/s 2024-04-25T16:44:25.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 613489 11.33 MB/sec execute 151 sec latency 16.670 ms 2024-04-25T16:44:26.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 615349 11.33 MB/sec execute 152 sec latency 46.940 ms 2024-04-25T16:44:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:27 smithi116 ceph-mon[32712]: pgmap v183: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2024-04-25T16:44:27.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:27 smithi112 ceph-mon[29122]: pgmap v183: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2024-04-25T16:44:27.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:27 smithi133 ceph-mon[32691]: pgmap v183: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 316 op/s 2024-04-25T16:44:27.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 617799 11.34 MB/sec execute 153 sec latency 27.196 ms 2024-04-25T16:44:28.879 INFO:tasks.workunit.client.0.smithi112.stdout: 1 619542 11.33 MB/sec execute 154 sec latency 14.176 ms 2024-04-25T16:44:29.501 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:29 smithi133 ceph-mon[32691]: pgmap v184: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-04-25T16:44:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:29 smithi116 ceph-mon[32712]: pgmap v184: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-04-25T16:44:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:29 smithi112 ceph-mon[29122]: pgmap v184: 97 pgs: 97 active+clean; 67 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-04-25T16:44:29.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 621913 11.35 MB/sec execute 155 sec latency 17.025 ms 2024-04-25T16:44:30.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 623975 11.32 MB/sec execute 156 sec latency 14.984 ms 2024-04-25T16:44:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:31 smithi116 ceph-mon[32712]: pgmap v185: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.5 MiB/s wr, 393 op/s 2024-04-25T16:44:31.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:31 smithi112 ceph-mon[29122]: pgmap v185: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.5 MiB/s wr, 393 op/s 2024-04-25T16:44:31.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:31 smithi133 ceph-mon[32691]: pgmap v185: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.5 MiB/s wr, 393 op/s 2024-04-25T16:44:31.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 625836 11.33 MB/sec execute 157 sec latency 48.959 ms 2024-04-25T16:44:32.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 628055 11.34 MB/sec execute 158 sec latency 48.575 ms 2024-04-25T16:44:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:33 smithi116 ceph-mon[32712]: pgmap v186: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-04-25T16:44:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:33 smithi112 ceph-mon[29122]: pgmap v186: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-04-25T16:44:33.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:33 smithi133 ceph-mon[32691]: pgmap v186: 97 pgs: 97 active+clean; 73 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-04-25T16:44:33.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 629855 11.32 MB/sec execute 159 sec latency 14.175 ms 2024-04-25T16:44:34.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 632391 11.35 MB/sec execute 160 sec latency 19.232 ms 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 -- 172.21.15.112:0/3474694193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c108f00 msgr2=0x7fe63c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 --2- 172.21.15.112:0/3474694193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c108f00 0x7fe63c102b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe630009900 tx=0x7fe63002a410 comp rx=0 tx=0).stop 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 -- 172.21.15.112:0/3474694193 shutdown_connections 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 --2- 172.21.15.112:0/3474694193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c1031c0 0x7fe63c10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 --2- 172.21.15.112:0/3474694193 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c108f00 0x7fe63c102b50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 --2- 172.21.15.112:0/3474694193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe63c108a60 0x7fe63c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 -- 172.21.15.112:0/3474694193 >> 172.21.15.112:0/3474694193 conn(0x7fe63c0781b0 msgr2=0x7fe63c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 -- 172.21.15.112:0/3474694193 shutdown_connections 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.893+0000 7fe641a34640 1 -- 172.21.15.112:0/3474694193 wait complete. 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 Processor -- start 2024-04-25T16:44:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 -- start start 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 0x7fe63c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe63c108a60 0x7fe63c19acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c108f00 0x7fe63c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c112c80 con 0x7fe63c1031c0 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.894+0000 7fe641a34640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c112b00 con 0x7fe63c108f00 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe641a34640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c112e00 con 0x7fe63c108a60 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 0x7fe63c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c108f00 0x7fe63c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 0x7fe63c19a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38342/0 (socket says 172.21.15.112:38342) 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 -- 172.21.15.112:0/4218001491 learned_addr learned my addr 172.21.15.112:0/4218001491 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe63c108a60 0x7fe63c19acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 -- 172.21.15.112:0/4218001491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe63c108a60 msgr2=0x7fe63c19acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe63c108a60 0x7fe63c19acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 -- 172.21.15.112:0/4218001491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c108f00 msgr2=0x7fe63c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c108f00 0x7fe63c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 -- 172.21.15.112:0/4218001491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe63c19f5d0 con 0x7fe63c1031c0 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c108f00 0x7fe63c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe63affd640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 0x7fe63c19a770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c970 tx=0x7fe62c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe640a32640 1 -- 172.21.15.112:0/4218001491 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c004220 con 0x7fe63c1031c0 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe63c19f8c0 con 0x7fe63c1031c0 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe63c1a7240 con 0x7fe63c1031c0 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe640a32640 1 -- 172.21.15.112:0/4218001491 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c0043c0 con 0x7fe63c1031c0 2024-04-25T16:44:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.895+0000 7fe640a32640 1 -- 172.21.15.112:0/4218001491 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c004a00 con 0x7fe63c1031c0 2024-04-25T16:44:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe640a32640 1 -- 172.21.15.112:0/4218001491 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fe62c004ba0 con 0x7fe63c1031c0 2024-04-25T16:44:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 Processor -- start 2024-04-25T16:44:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 -- start start 2024-04-25T16:44:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162066f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161c9a2f0 0x55d162067440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162067980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d161fc9460 con 0x55d161cc31b0 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d161f83590 con 0x55d161ca24d0 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe6489c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d161fc5420 con 0x55d161c9a2f0 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162066f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38346/0 (socket says 172.21.15.112:38346) 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe63b7fe640 1 -- 172.21.15.112:0/4094180119 learned_addr learned my addr 172.21.15.112:0/4094180119 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162067980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.898+0000 7fe640a32640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fe6240afc50 0x7fe6240b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63affd640 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161c9a2f0 0x55d162067440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 -- 172.21.15.112:0/4094180119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161c9a2f0 msgr2=0x55d162067440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161c9a2f0 0x55d162067440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 -- 172.21.15.112:0/4094180119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 msgr2=0x55d162067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162067980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 -- 172.21.15.112:0/4094180119 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d161f900d0 con 0x55d161cc31b0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fe6240afc50 0x7fe6240b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe640a32640 1 -- 172.21.15.112:0/4218001491 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fe62c0ded50 con 0x7fe63c1031c0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162066f00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe634011940 tx=0x7fe634011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6260891ed613eb78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe621ffb640 1 -- 172.21.15.112:0/4094180119 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634017070 con 0x55d161cc31b0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe621ffb640 1 -- 172.21.15.112:0/4094180119 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe634011250 con 0x55d161cc31b0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe621ffb640 1 -- 172.21.15.112:0/4094180119 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe634010b30 con 0x55d161cc31b0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.899+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fe6240afc50 0x7fe6240b2160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe634023000 tx=0x7fe634024000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 -- 172.21.15.112:0/4094180119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 msgr2=0x55d162066f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162066f00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe634011940 tx=0x7fe634011e00 comp rx=0 tx=0).stop 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 -- 172.21.15.112:0/4094180119 shutdown_connections 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161c9a2f0 0x55d162067440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162067980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 --2- 172.21.15.112:0/4094180119 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162066f00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 -- 172.21.15.112:0/4094180119 >> 172.21.15.112:0/4094180119 conn(0x55d161cb7300 msgr2=0x55d1620636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 -- 172.21.15.112:0/4094180119 shutdown_connections 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.900+0000 7fe6489c4740 1 -- 172.21.15.112:0/4094180119 wait complete. 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 -- start start 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 Processor -- start 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d1620715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162071b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162077520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d162074030 con 0x55d161cc31b0 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d161f50b60 con 0x55d161c9a2f0 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1620741b0 con 0x55d161ca24d0 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe6489c4740 10 client.0 _ll_register_callbacks cb 0x55d161f752b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe63affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d1620715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162077520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe63affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d1620715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:44218/0 (socket says 172.21.15.112:44218) 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe63affd640 1 -- 172.21.15.112:0/1352562377 learned_addr learned my addr 172.21.15.112:0/1352562377 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.901+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162071b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe63affd640 1 -- 172.21.15.112:0/1352562377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 msgr2=0x55d162071b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe63affd640 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162071b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe63affd640 1 -- 172.21.15.112:0/1352562377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 msgr2=0x55d162077520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe63affd640 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162077520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe63affd640 1 -- 172.21.15.112:0/1352562377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d161cdb210 con 0x55d161c9a2f0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe63affd640 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d1620715c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0a6f10 tx=0x7fe62c00af60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6a9bc278884dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe61affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe61affd640 1 -- 172.21.15.112:0/1352562377 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c01d050 con 0x55d161c9a2f0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe61affd640 1 -- 172.21.15.112:0/1352562377 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fe62c004e90 con 0x55d161c9a2f0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe634011250 con 0x55d161c9a2f0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe61affd640 1 -- 172.21.15.112:0/1352562377 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe62c0a66f0 con 0x55d161c9a2f0 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe6489c4740 10 client.25583 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d161da7620 con 0x55d161c9a2f0 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe61affd640 1 -- 172.21.15.112:0/1352562377 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe634011250 con 0x55d161c9a2f0 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe61affd640 1 -- 172.21.15.112:0/1352562377 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe62c0a28f0 con 0x55d161c9a2f0 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe6489c4740 10 client.25583 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.902+0000 7fe6489c4740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe6489c4740 20 client.25583 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe6489c4740 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c81d80 0x55d162083b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe6489c4740 4 client.25583 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d161fd7070 con 0x55d161c81d80 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c81d80 0x55d162083b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe61affd640 10 client.25583 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.903+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c81d80 0x55d162083b30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3371ea782db4c8 server_cookie=3932c60a84055c81 in_seq=0 out_seq=0 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe61affd640 1 -- 172.21.15.112:0/1352562377 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d161fd7070 con 0x55d161c81d80 2024-04-25T16:44:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe61affd640 10 client.25583 handle_command_reply: tid=0 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 client.25583 shutdown 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 msgr2=0x55d1620715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d1620715c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0a6f10 tx=0x7fe62c00af60 comp rx=0 tx=0).stop 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 shutdown_connections 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c81d80 0x55d162083b30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d162077520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe61affd640 0 client.25583 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 --2- 172.21.15.112:0/1352562377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d1620715c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 >> 172.21.15.112:0/1352562377 conn(0x55d161cb7300 msgr2=0x55d162066d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 shutdown_connections 2024-04-25T16:44:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 1 -- 172.21.15.112:0/1352562377 wait complete. 2024-04-25T16:44:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.905+0000 7fe6489c4740 20 client.25583 trim_cache size 0 max 16384 2024-04-25T16:44:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 Processor -- start 2024-04-25T16:44:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 -- start start 2024-04-25T16:44:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d162067ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162077420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d16207dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1b0100 con 0x55d161cc31b0 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1ad9d0 con 0x55d161c9a2f0 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe6489c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1ab840 con 0x55d161ca24d0 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d16207dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe63affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d162067ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe63affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d162067ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:44234/0 (socket says 172.21.15.112:44234) 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d16207dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38360/0 (socket says 172.21.15.112:38360) 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe63affd640 1 -- 172.21.15.112:0/2423076811 learned_addr learned my addr 172.21.15.112:0/2423076811 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.921+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162077420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe63affd640 1 -- 172.21.15.112:0/2423076811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 msgr2=0x55d162077420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe63affd640 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162077420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe63affd640 1 -- 172.21.15.112:0/2423076811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 msgr2=0x55d16207dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe63affd640 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d16207dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe63affd640 1 -- 172.21.15.112:0/2423076811 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d16206cf70 con 0x55d161c9a2f0 2024-04-25T16:44:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe63affd640 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d162067ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0a3000 tx=0x7fe62c0a36a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec8653e51bfa42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe6217fa640 1 -- 172.21.15.112:0/2423076811 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c01d070 con 0x55d161c9a2f0 2024-04-25T16:44:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe6217fa640 1 -- 172.21.15.112:0/2423076811 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c0c9040 con 0x55d161c9a2f0 2024-04-25T16:44:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.922+0000 7fe6217fa640 1 -- 172.21.15.112:0/2423076811 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0a5eb0 con 0x55d161c9a2f0 2024-04-25T16:44:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 -- 172.21.15.112:0/2423076811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 msgr2=0x55d162067ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d162067ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0a3000 tx=0x7fe62c0a36a0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 -- 172.21.15.112:0/2423076811 shutdown_connections 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161ca24d0 0x55d162077420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161cc31b0 0x55d16207dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 --2- 172.21.15.112:0/2423076811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161c9a2f0 0x55d162067ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 -- 172.21.15.112:0/2423076811 >> 172.21.15.112:0/2423076811 conn(0x55d161cb7300 msgr2=0x55d16205d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 -- 172.21.15.112:0/2423076811 shutdown_connections 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 -- 172.21.15.112:0/2423076811 wait complete. 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 -- start start 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.923+0000 7fe6489c4740 1 Processor -- start 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 0x55d16207dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162078fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161cc31b0 0x55d1620794e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1abe90 con 0x55d161c9a2f0 2024-04-25T16:44:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1ad300 con 0x55d161ca24d0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe63c1ac560 con 0x55d161cc31b0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 10 client.0 _ll_register_callbacks cb 0x7fe62c00c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 0x55d16207dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 0x55d16207dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38364/0 (socket says 172.21.15.112:38364) 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 -- 172.21.15.112:0/455594393 learned_addr learned my addr 172.21.15.112:0/455594393 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162078fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63b7fe640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161cc31b0 0x55d1620794e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 -- 172.21.15.112:0/455594393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161cc31b0 msgr2=0x55d1620794e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161cc31b0 0x55d1620794e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 -- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 msgr2=0x55d162078fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162078fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 -- 172.21.15.112:0/455594393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d16207e400 con 0x55d161c9a2f0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe63affd640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 0x55d16207dc50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0a41f0 tx=0x7fe62c0dc560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a680b76dff7fb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe61affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:44:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe61affd640 1 -- 172.21.15.112:0/455594393 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c01d040 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe61affd640 1 -- 172.21.15.112:0/455594393 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7fe62c0a5eb0 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.924+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe62c0c9040 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe61affd640 1 -- 172.21.15.112:0/455594393 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe62c0056a0 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 10 client.15891 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d1620ebb70 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe61affd640 1 -- 172.21.15.112:0/455594393 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe62c0c9040 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe61affd640 1 -- 172.21.15.112:0/455594393 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fe62c0ac410 con 0x55d161c9a2f0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 10 client.15891 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 20 client.15891 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c571c0 0x55d1620ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 4 client.15891 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d161fd0ee0 con 0x55d161c571c0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.925+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c571c0 0x55d1620ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.926+0000 7fe61affd640 10 client.15891 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:34.926+0000 7fe63a7fc640 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c571c0 0x55d1620ee110 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88e1217ec57439e server_cookie=e68a4215e3cbd953 in_seq=0 out_seq=0 2024-04-25T16:44:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.081+0000 7fe61affd640 1 -- 172.21.15.112:0/455594393 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d161fd0ee0 con 0x55d161c571c0 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.081+0000 7fe61affd640 10 client.15891 handle_command_reply: tid=0 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.081+0000 7fe6489c4740 1 client.15891 shutdown 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.081+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 msgr2=0x55d16207dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.081+0000 7fe6489c4740 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 0x55d16207dc50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe62c0a41f0 tx=0x7fe62c0dc560 comp rx=0 tx=0).stop 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 shutdown_connections 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d161c571c0 0x55d1620ee110 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe61affd640 0 client.15891 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d161cc31b0 0x55d1620794e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d161ca24d0 0x55d162078fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 --2- 172.21.15.112:0/455594393 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d161c9a2f0 0x55d16207dc50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 >> 172.21.15.112:0/455594393 conn(0x55d161cb7300 msgr2=0x55d16205e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 shutdown_connections 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 1 -- 172.21.15.112:0/455594393 wait complete. 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.082+0000 7fe6489c4740 20 client.15891 trim_cache size 0 max 16384 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:44:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:44:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fe6240afc50 msgr2=0x7fe6240b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fe6240afc50 0x7fe6240b2160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe634023000 tx=0x7fe634024000 comp rx=0 tx=0).stop 2024-04-25T16:44:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 msgr2=0x7fe63c19a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 0x7fe63c19a770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c970 tx=0x7fe62c00ce30 comp rx=0 tx=0).stop 2024-04-25T16:44:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 shutdown_connections 2024-04-25T16:44:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fe6240afc50 0x7fe6240b2160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe63c108a60 0x7fe63c19acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe63c108f00 0x7fe63c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 --2- 172.21.15.112:0/4218001491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe63c1031c0 0x7fe63c19a770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 >> 172.21.15.112:0/4218001491 conn(0x7fe63c0781b0 msgr2=0x7fe63c06a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 shutdown_connections 2024-04-25T16:44:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:35.083+0000 7fe641a34640 1 -- 172.21.15.112:0/4218001491 wait complete. 2024-04-25T16:44:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:35 smithi116 ceph-mon[32712]: pgmap v187: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 374 op/s 2024-04-25T16:44:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:35 smithi112 ceph-mon[29122]: pgmap v187: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 374 op/s 2024-04-25T16:44:35.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:35 smithi133 ceph-mon[32691]: pgmap v187: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.2 MiB/s wr, 374 op/s 2024-04-25T16:44:35.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 633725 11.32 MB/sec execute 161 sec latency 93.181 ms 2024-04-25T16:44:36.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 635936 11.33 MB/sec execute 162 sec latency 45.716 ms 2024-04-25T16:44:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:37 smithi116 ceph-mon[32712]: pgmap v188: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-04-25T16:44:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:37 smithi112 ceph-mon[29122]: pgmap v188: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-04-25T16:44:37.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:37 smithi133 ceph-mon[32691]: pgmap v188: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 250 op/s 2024-04-25T16:44:37.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 637571 11.30 MB/sec execute 163 sec latency 50.234 ms 2024-04-25T16:44:38.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:38 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:44:38.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:38 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:44:38.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:38 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:44:38.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 639681 11.31 MB/sec execute 164 sec latency 93.157 ms 2024-04-25T16:44:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:39 smithi116 ceph-mon[32712]: pgmap v189: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-04-25T16:44:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:39 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:44:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:39 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:44:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:39 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:44:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:39 smithi112 ceph-mon[29122]: pgmap v189: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-04-25T16:44:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:39 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:44:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:39 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:44:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:39 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:44:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:39 smithi133 ceph-mon[32691]: pgmap v189: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-04-25T16:44:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:39 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:44:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:39 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:44:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:39 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:44:39.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 641858 11.29 MB/sec execute 165 sec latency 14.494 ms 2024-04-25T16:44:40.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 643955 11.30 MB/sec execute 166 sec latency 15.962 ms 2024-04-25T16:44:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:41 smithi116 ceph-mon[32712]: pgmap v190: 97 pgs: 97 active+clean; 70 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 396 op/s 2024-04-25T16:44:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:41 smithi112 ceph-mon[29122]: pgmap v190: 97 pgs: 97 active+clean; 70 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 396 op/s 2024-04-25T16:44:41.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:41 smithi133 ceph-mon[32691]: pgmap v190: 97 pgs: 97 active+clean; 70 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.4 MiB/s wr, 396 op/s 2024-04-25T16:44:41.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 646485 11.33 MB/sec execute 167 sec latency 22.518 ms 2024-04-25T16:44:42.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 647959 11.30 MB/sec execute 168 sec latency 92.124 ms 2024-04-25T16:44:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:43 smithi116 ceph-mon[32712]: pgmap v191: 97 pgs: 97 active+clean; 70 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-04-25T16:44:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:43 smithi112 ceph-mon[29122]: pgmap v191: 97 pgs: 97 active+clean; 70 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-04-25T16:44:43.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:43 smithi133 ceph-mon[32691]: pgmap v191: 97 pgs: 97 active+clean; 70 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-04-25T16:44:43.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 650293 11.32 MB/sec execute 169 sec latency 15.185 ms 2024-04-25T16:44:44.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 652316 11.29 MB/sec execute 170 sec latency 16.344 ms 2024-04-25T16:44:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:45 smithi116 ceph-mon[32712]: pgmap v192: 97 pgs: 97 active+clean; 76 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2024-04-25T16:44:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:45 smithi112 ceph-mon[29122]: pgmap v192: 97 pgs: 97 active+clean; 76 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2024-04-25T16:44:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:45 smithi133 ceph-mon[32691]: pgmap v192: 97 pgs: 97 active+clean; 76 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.1 MiB/s wr, 380 op/s 2024-04-25T16:44:45.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 654200 11.30 MB/sec execute 171 sec latency 17.542 ms 2024-04-25T16:44:46.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 656706 11.31 MB/sec execute 172 sec latency 15.473 ms 2024-04-25T16:44:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:47 smithi116 ceph-mon[32712]: pgmap v193: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-04-25T16:44:47.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:47 smithi112 ceph-mon[29122]: pgmap v193: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-04-25T16:44:47.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:47 smithi133 ceph-mon[32691]: pgmap v193: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.0 MiB/s wr, 313 op/s 2024-04-25T16:44:47.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 658051 11.29 MB/sec execute 173 sec latency 65.693 ms 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.185+0000 7f46f23f3640 1 -- 172.21.15.112:0/3345457529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 msgr2=0x7f46ec10f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.185+0000 7f46f23f3640 1 --2- 172.21.15.112:0/3345457529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 0x7f46ec10f570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009ca0 tx=0x7f46e402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.185+0000 7f46f23f3640 1 -- 172.21.15.112:0/3345457529 shutdown_connections 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 --2- 172.21.15.112:0/3345457529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 0x7f46ec10f570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 --2- 172.21.15.112:0/3345457529 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec1072c0 0x7f46ec1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 --2- 172.21.15.112:0/3345457529 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ec103400 0x7f46ec103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 -- 172.21.15.112:0/3345457529 >> 172.21.15.112:0/3345457529 conn(0x7f46ec0fd8a0 msgr2=0x7f46ec0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:48.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 -- 172.21.15.112:0/3345457529 shutdown_connections 2024-04-25T16:44:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 -- 172.21.15.112:0/3345457529 wait complete. 2024-04-25T16:44:48.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 Processor -- start 2024-04-25T16:44:48.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.186+0000 7f46f23f3640 1 -- start start 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f23f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 0x7f46ec19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f23f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ec1072c0 0x7f46ec19bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f23f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 0x7f46ec19c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f23f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec112c60 con 0x7f46ec103400 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f23f3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec112ae0 con 0x7f46ec107d40 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f23f3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec112de0 con 0x7f46ec1072c0 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 0x7f46ec19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 0x7f46ec19b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37936/0 (socket says 172.21.15.112:37936) 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 -- 172.21.15.112:0/1767440791 learned_addr learned my addr 172.21.15.112:0/1767440791 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46f0990640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 0x7f46ec19c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ec1072c0 0x7f46ec19bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 -- 172.21.15.112:0/1767440791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ec1072c0 msgr2=0x7f46ec19bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ec1072c0 0x7f46ec19bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 -- 172.21.15.112:0/1767440791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 msgr2=0x7f46ec19c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 0x7f46ec19c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 -- 172.21.15.112:0/1767440791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1a68d0 con 0x7f46ec103400 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.187+0000 7f46ebfff640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 0x7f46ec19b5c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e9f0 tx=0x7f46dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.188+0000 7f46e97fa640 1 -- 172.21.15.112:0/1767440791 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00ccc0 con 0x7f46ec103400 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.188+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1a6bc0 con 0x7f46ec103400 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.188+0000 7f46e97fa640 1 -- 172.21.15.112:0/1767440791 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46dc0054d0 con 0x7f46ec103400 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.188+0000 7f46e97fa640 1 -- 172.21.15.112:0/1767440791 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc010430 con 0x7f46ec103400 2024-04-25T16:44:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.188+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a70c0 con 0x7f46ec103400 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 Processor -- start 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 -- start start 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46e97fa640 1 -- 172.21.15.112:0/1767440791 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f46dc0105d0 con 0x7f46ec103400 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a04f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a04f7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a04f7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605a045d460 con 0x5605a012e2f0 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5605a0417590 con 0x5605a01364d0 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46f9381740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5605a0459420 con 0x5605a01571b0 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a04f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.190+0000 7f46eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a04f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a04f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:36680/0 (socket says 172.21.15.112:36680) 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46f0990640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a04f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 -- 172.21.15.112:0/2908298602 learned_addr learned my addr 172.21.15.112:0/2908298602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a04f7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37946/0 (socket says 172.21.15.112:37946) 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 -- 172.21.15.112:0/2908298602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 msgr2=0x5605a04f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a04f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 -- 172.21.15.112:0/2908298602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 msgr2=0x5605a04f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a04f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 -- 172.21.15.112:0/2908298602 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a04240d0 con 0x5605a01364d0 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a04f7570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009ba0 tx=0x7f46e0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3412aeb21ccc4e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46e97fa640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46d40afc50 0x7f46d40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46f0990640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46d40afc50 0x7f46d40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46e97fa640 1 -- 172.21.15.112:0/1767440791 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f46dc0df3e0 con 0x7f46ec103400 2024-04-25T16:44:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46d27fc640 1 -- 172.21.15.112:0/2908298602 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0004230 con 0x5605a01364d0 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46d27fc640 1 -- 172.21.15.112:0/2908298602 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e0009e10 con 0x5605a01364d0 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.191+0000 7f46d27fc640 1 -- 172.21.15.112:0/2908298602 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e00196d0 con 0x5605a01364d0 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 -- 172.21.15.112:0/2908298602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 msgr2=0x5605a04f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f0990640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46d40afc50 0x7f46d40b2160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x5605a04f3c60 tx=0x7f46e402f310 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a04f7570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f46e0009ba0 tx=0x7f46e0011810 comp rx=0 tx=0).stop 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 -- 172.21.15.112:0/2908298602 shutdown_connections 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a04f7570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a04f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 --2- 172.21.15.112:0/2908298602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a04f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 -- 172.21.15.112:0/2908298602 >> 172.21.15.112:0/2908298602 conn(0x5605a014b300 msgr2=0x5605a04f32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 -- 172.21.15.112:0/2908298602 shutdown_connections 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.192+0000 7f46f9381740 1 -- 172.21.15.112:0/2908298602 wait complete. 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 -- start start 2024-04-25T16:44:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 Processor -- start 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a012e2f0 0x5605a0501140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 0x5605a0501680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01571b0 0x5605a05070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5605a0503c20 con 0x5605a01364d0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5605a03e4b60 con 0x5605a01571b0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5605a0503da0 con 0x5605a012e2f0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f9381740 10 client.0 _ll_register_callbacks cb 0x5605a04092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 0x5605a0501680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a012e2f0 0x5605a0501140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 0x5605a0501680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37956/0 (socket says 172.21.15.112:37956) 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46eb7fe640 1 -- 172.21.15.112:0/1915375383 learned_addr learned my addr 172.21.15.112:0/1915375383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.193+0000 7f46f0990640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01571b0 0x5605a05070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46eb7fe640 1 -- 172.21.15.112:0/1915375383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a012e2f0 msgr2=0x5605a0501140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.198 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a012e2f0 0x5605a0501140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46eb7fe640 1 -- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01571b0 msgr2=0x5605a05070a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01571b0 0x5605a05070a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46eb7fe640 1 -- 172.21.15.112:0/1915375383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a016f210 con 0x5605a01364d0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46ebfff640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a012e2f0 0x5605a0501140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 0x5605a0501680 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f46e0005bc0 tx=0x7f46e00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef5c9ec19628067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46cb7fe640 1 -- 172.21.15.112:0/1915375383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0028020 con 0x5605a01364d0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46cb7fe640 1 -- 172.21.15.112:0/1915375383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f46e0005340 con 0x5605a01364d0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e0009e10 con 0x5605a01364d0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46cb7fe640 1 -- 172.21.15.112:0/1915375383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f46e0033050 con 0x5605a01364d0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46f9381740 10 client.15903 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5605a023b620 con 0x5605a01364d0 2024-04-25T16:44:48.199 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46cb7fe640 1 -- 172.21.15.112:0/1915375383 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e0009e10 con 0x5605a01364d0 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46cb7fe640 1 -- 172.21.15.112:0/1915375383 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f46e0018070 con 0x5605a01364d0 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46f9381740 10 client.15903 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46f9381740 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.194+0000 7f46f9381740 20 client.15903 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.195+0000 7f46f9381740 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a0115d80 0x5605a0513590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.195+0000 7f46f9381740 4 client.15903 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.195+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605a046b070 con 0x5605a0115d80 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.195+0000 7f46ebfff640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a0115d80 0x5605a0513590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.195+0000 7f46cb7fe640 10 client.15903 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.195+0000 7f46ebfff640 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a0115d80 0x5605a0513590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ffc3ba45191adc server_cookie=df1baf6ca8c8f1da in_seq=0 out_seq=0 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46cb7fe640 1 -- 172.21.15.112:0/1915375383 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5605a046b070 con 0x5605a0115d80 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46cb7fe640 10 client.15903 handle_command_reply: tid=0 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 client.15903 shutdown 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 msgr2=0x5605a0501680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.200 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 0x5605a0501680 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f46e0005bc0 tx=0x7f46e00098d0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 shutdown_connections 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01571b0 0x5605a05070a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01364d0 0x5605a0501680 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a0115d80 0x5605a0513590 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 --2- 172.21.15.112:0/1915375383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a012e2f0 0x5605a0501140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46cb7fe640 0 client.15903 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.196+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 >> 172.21.15.112:0/1915375383 conn(0x5605a014b300 msgr2=0x5605a04f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.197+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 shutdown_connections 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.197+0000 7f46f9381740 1 -- 172.21.15.112:0/1915375383 wait complete. 2024-04-25T16:44:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.197+0000 7f46f9381740 20 client.15903 trim_cache size 0 max 16384 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 Processor -- start 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 -- start start 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a012e2f0 0x5605a0507010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01364d0 0x5605a0503080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 0x5605a04f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1b23a0 con 0x5605a01571b0 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1afc70 con 0x5605a012e2f0 2024-04-25T16:44:48.211 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f9381740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1adac0 con 0x5605a01364d0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 0x5605a04f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a012e2f0 0x5605a0507010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 0x5605a04f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37968/0 (socket says 172.21.15.112:37968) 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 -- 172.21.15.112:0/3323759095 learned_addr learned my addr 172.21.15.112:0/3323759095 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01364d0 0x5605a0503080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 -- 172.21.15.112:0/3323759095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01364d0 msgr2=0x5605a0503080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01364d0 0x5605a0503080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 -- 172.21.15.112:0/3323759095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a012e2f0 msgr2=0x5605a0507010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a012e2f0 0x5605a0507010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46f0990640 1 -- 172.21.15.112:0/3323759095 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605a04f3e90 con 0x5605a01571b0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.212+0000 7f46ebfff640 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a012e2f0 0x5605a0507010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.213+0000 7f46f0990640 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 0x5605a04f7080 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f46e4035000 tx=0x7f46e4044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600b9384afd71c90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.213+0000 7f46d1ffb640 1 -- 172.21.15.112:0/3323759095 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4003a30 con 0x5605a01571b0 2024-04-25T16:44:48.212 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.213+0000 7f46d1ffb640 1 -- 172.21.15.112:0/3323759095 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e4036050 con 0x5605a01571b0 2024-04-25T16:44:48.213 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.213+0000 7f46d1ffb640 1 -- 172.21.15.112:0/3323759095 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e402f8c0 con 0x5605a01571b0 2024-04-25T16:44:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- 172.21.15.112:0/3323759095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 msgr2=0x5605a04f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 0x5605a04f7080 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f46e4035000 tx=0x7f46e4044000 comp rx=0 tx=0).stop 2024-04-25T16:44:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- 172.21.15.112:0/3323759095 shutdown_connections 2024-04-25T16:44:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01364d0 0x5605a0503080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a01571b0 0x5605a04f7080 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.216 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- 172.21.15.112:0/3323759095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a012e2f0 0x5605a0507010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- 172.21.15.112:0/3323759095 >> 172.21.15.112:0/3323759095 conn(0x5605a014b300 msgr2=0x5605a04f6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- 172.21.15.112:0/3323759095 shutdown_connections 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- 172.21.15.112:0/3323759095 wait complete. 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- start start 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 Processor -- start 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a0573b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a05740c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a0508c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1ae130 con 0x5605a012e2f0 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1af5a0 con 0x5605a01364d0 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.214+0000 7f46f9381740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1ae800 con 0x5605a01571b0 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46f9381740 10 client.0 _ll_register_callbacks cb 0x7f46dc0e2e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a0573b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.217 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a0573b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37974/0 (socket says 172.21.15.112:37974) 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 -- 172.21.15.112:0/3456012196 learned_addr learned my addr 172.21.15.112:0/3456012196 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46f0990640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a0508c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a05740c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 -- 172.21.15.112:0/3456012196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 msgr2=0x5605a0508c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a0508c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 -- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 msgr2=0x5605a05740c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a05740c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 -- 172.21.15.112:0/3456012196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605a0509280 con 0x5605a012e2f0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46ebfff640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a0573b80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e9c0 tx=0x7f46dc0df770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e875f99190d35924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46cb7fe640 1 -- 172.21.15.112:0/3456012196 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0a6cd0 con 0x5605a012e2f0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46cb7fe640 1 -- 172.21.15.112:0/3456012196 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f46dc0046c0 con 0x5605a012e2f0 2024-04-25T16:44:48.218 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e4036050 con 0x5605a012e2f0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46cb7fe640 1 -- 172.21.15.112:0/3456012196 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f46dc0a7070 con 0x5605a012e2f0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46f9381740 10 client.15909 fetch_fsmap learned FSMap version 23 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5605a057f620 con 0x5605a012e2f0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.215+0000 7f46cb7fe640 1 -- 172.21.15.112:0/3456012196 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e4036050 con 0x5605a012e2f0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46cb7fe640 1 -- 172.21.15.112:0/3456012196 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f46dc0aff30 con 0x5605a012e2f0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46f9381740 10 client.15909 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46f9381740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46f9381740 20 client.15909 populate_metadata read hostname 'smithi112' 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46f9381740 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a00eb1c0 0x5605a0581d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46f9381740 4 client.15909 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605a0464ee0 con 0x5605a00eb1c0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a00eb1c0 0x5605a0581d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46cb7fe640 10 client.15909 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:48.219 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.216+0000 7f46eb7fe640 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a00eb1c0 0x5605a0581d80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7abe6e2aed6d9f94 server_cookie=ea28f387a4787dfe in_seq=0 out_seq=0 2024-04-25T16:44:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.368+0000 7f46cb7fe640 1 -- 172.21.15.112:0/3456012196 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605a0464ee0 con 0x5605a00eb1c0 2024-04-25T16:44:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.368+0000 7f46cb7fe640 10 client.15909 handle_command_reply: tid=0 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 client.15909 shutdown 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 msgr2=0x5605a0573b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a0573b80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e9c0 tx=0x7f46dc0df770 comp rx=0 tx=0).stop 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 shutdown_connections 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5605a00eb1c0 0x5605a0581d80 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605a01571b0 0x5605a0508c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5605a01364d0 0x5605a05740c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 --2- 172.21.15.112:0/3456012196 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5605a012e2f0 0x5605a0573b80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 >> 172.21.15.112:0/3456012196 conn(0x5605a014b300 msgr2=0x5605a04f73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46cb7fe640 0 client.15909 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 shutdown_connections 2024-04-25T16:44:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 1 -- 172.21.15.112:0/3456012196 wait complete. 2024-04-25T16:44:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.369+0000 7f46f9381740 20 client.15909 trim_cache size 0 max 16384 2024-04-25T16:44:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:44:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:44:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:44:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46d40afc50 msgr2=0x7f46d40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46d40afc50 0x7f46d40b2160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x5605a04f3c60 tx=0x7f46e402f310 comp rx=0 tx=0).stop 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 msgr2=0x7f46ec19b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 0x7f46ec19b5c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f46dc00e9f0 tx=0x7f46dc00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 shutdown_connections 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46d40afc50 0x7f46d40b2160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ec1072c0 0x7f46ec19bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46ec107d40 0x7f46ec19c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 --2- 172.21.15.112:0/1767440791 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46ec103400 0x7f46ec19b5c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 >> 172.21.15.112:0/1767440791 conn(0x7f46ec0fd8a0 msgr2=0x7f46ec105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 shutdown_connections 2024-04-25T16:44:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:44:48.370+0000 7f46f23f3640 1 -- 172.21.15.112:0/1767440791 wait complete. 2024-04-25T16:44:48.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 660251 11.29 MB/sec execute 174 sec latency 102.768 ms 2024-04-25T16:44:49.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:49 smithi116 ceph-mon[32712]: pgmap v194: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-04-25T16:44:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:49 smithi112 ceph-mon[29122]: pgmap v194: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-04-25T16:44:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:49 smithi133 ceph-mon[32691]: pgmap v194: 97 pgs: 97 active+clean; 77 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 302 op/s 2024-04-25T16:44:49.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 662023 11.28 MB/sec execute 175 sec latency 16.048 ms 2024-04-25T16:44:50.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 664369 11.30 MB/sec execute 176 sec latency 17.096 ms 2024-04-25T16:44:51.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:51 smithi116 ceph-mon[32712]: pgmap v195: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-04-25T16:44:51.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:51 smithi112 ceph-mon[29122]: pgmap v195: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-04-25T16:44:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:51 smithi133 ceph-mon[32691]: pgmap v195: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.3 MiB/s wr, 493 op/s 2024-04-25T16:44:51.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 666055 11.27 MB/sec execute 177 sec latency 17.179 ms 2024-04-25T16:44:52.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 667994 11.28 MB/sec execute 178 sec latency 55.572 ms 2024-04-25T16:44:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:53 smithi116 ceph-mon[32712]: pgmap v196: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 286 op/s 2024-04-25T16:44:53.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:53 smithi112 ceph-mon[29122]: pgmap v196: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 286 op/s 2024-04-25T16:44:53.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:53 smithi133 ceph-mon[32691]: pgmap v196: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 286 op/s 2024-04-25T16:44:53.880 INFO:tasks.workunit.client.0.smithi112.stdout: 1 670012 11.26 MB/sec execute 179 sec latency 38.122 ms 2024-04-25T16:44:54.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 671952 11.26 MB/sec execute 180 sec latency 17.582 ms 2024-04-25T16:44:55.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:55 smithi116 ceph-mon[32712]: pgmap v197: 97 pgs: 97 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 343 op/s 2024-04-25T16:44:55.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:55 smithi112 ceph-mon[29122]: pgmap v197: 97 pgs: 97 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 343 op/s 2024-04-25T16:44:55.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:55 smithi133 ceph-mon[32691]: pgmap v197: 97 pgs: 97 active+clean; 69 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 343 op/s 2024-04-25T16:44:55.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 674577 11.28 MB/sec execute 181 sec latency 26.137 ms 2024-04-25T16:44:56.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 676296 11.27 MB/sec execute 182 sec latency 14.497 ms 2024-04-25T16:44:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:57 smithi112 ceph-mon[29122]: pgmap v198: 97 pgs: 97 active+clean; 73 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-04-25T16:44:57.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:57 smithi133 ceph-mon[32691]: pgmap v198: 97 pgs: 97 active+clean; 73 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-04-25T16:44:57.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:57 smithi116 ceph-mon[32712]: pgmap v198: 97 pgs: 97 active+clean; 73 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-04-25T16:44:57.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 678532 11.28 MB/sec execute 183 sec latency 59.326 ms 2024-04-25T16:44:58.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 680088 11.25 MB/sec execute 184 sec latency 23.690 ms 2024-04-25T16:44:59.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:44:59 smithi112 ceph-mon[29122]: pgmap v199: 97 pgs: 97 active+clean; 73 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-04-25T16:44:59.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:44:59 smithi133 ceph-mon[32691]: pgmap v199: 97 pgs: 97 active+clean; 73 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-04-25T16:44:59.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:44:59 smithi116 ceph-mon[32712]: pgmap v199: 97 pgs: 97 active+clean; 73 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 289 op/s 2024-04-25T16:44:59.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 682414 11.27 MB/sec execute 185 sec latency 15.510 ms 2024-04-25T16:45:00.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 684142 11.24 MB/sec execute 186 sec latency 26.622 ms 2024-04-25T16:45:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 -- 172.21.15.112:0/2207178462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8158109bf0 msgr2=0x7f8158101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 --2- 172.21.15.112:0/2207178462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8158109bf0 0x7f8158101b60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8148009900 tx=0x7f814802a410 comp rx=0 tx=0).stop 2024-04-25T16:45:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 -- 172.21.15.112:0/2207178462 shutdown_connections 2024-04-25T16:45:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 --2- 172.21.15.112:0/2207178462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f81581024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.469 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 --2- 172.21.15.112:0/2207178462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815810e5a0 0x7f81581109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 --2- 172.21.15.112:0/2207178462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8158109bf0 0x7f8158101b60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 -- 172.21.15.112:0/2207178462 >> 172.21.15.112:0/2207178462 conn(0x7f81580778b0 msgr2=0x7f8158077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 -- 172.21.15.112:0/2207178462 shutdown_connections 2024-04-25T16:45:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.470+0000 7f815ca61640 1 -- 172.21.15.112:0/2207178462 wait complete. 2024-04-25T16:45:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 Processor -- start 2024-04-25T16:45:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 -- start start 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f815819a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8158109bf0 0x7f815819ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 0x7f815819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8158113d60 con 0x7f815810e5a0 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8158113be0 con 0x7f8158109bf0 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.471+0000 7f815ca61640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8158113ee0 con 0x7f81581020a0 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 0x7f815819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f815659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f815819a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 0x7f815819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40346/0 (socket says 172.21.15.112:40346) 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f815659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f815819a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:49616/0 (socket says 172.21.15.112:49616) 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 -- 172.21.15.112:0/1309512677 learned_addr learned my addr 172.21.15.112:0/1309512677 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:01.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f814ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8158109bf0 0x7f815819ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 -- 172.21.15.112:0/1309512677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 msgr2=0x7f815819a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f815819a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 -- 172.21.15.112:0/1309512677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8158109bf0 msgr2=0x7f815819ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8158109bf0 0x7f815819ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 -- 172.21.15.112:0/1309512677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815819f600 con 0x7f815810e5a0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f815659c640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f815819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f8156d9d640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 0x7f815819efc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f815000eb40 tx=0x7f815000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f814f7fe640 1 -- 172.21.15.112:0/1309512677 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150004b20 con 0x7f815810e5a0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f814f7fe640 1 -- 172.21.15.112:0/1309512677 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8150004cc0 con 0x7f815810e5a0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815819f8f0 con 0x7f815810e5a0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f814f7fe640 1 -- 172.21.15.112:0/1309512677 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150010720 con 0x7f815810e5a0 2024-04-25T16:45:01.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.472+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815819fdf0 con 0x7f815810e5a0 2024-04-25T16:45:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 Processor -- start 2024-04-25T16:45:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f814f7fe640 1 -- 172.21.15.112:0/1309512677 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f8150005560 con 0x7f815810e5a0 2024-04-25T16:45:01.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 -- start start 2024-04-25T16:45:01.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a24696af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a24697030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a24697570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a245fd460 con 0x560a242f71b0 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a245b7590 con 0x560a242d64d0 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f81639ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a245f9420 con 0x560a242ce2f0 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f814ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a24696af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f814ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a24696af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40362/0 (socket says 172.21.15.112:40362) 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f814ffff640 1 -- 172.21.15.112:0/108140010 learned_addr learned my addr 172.21.15.112:0/108140010 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.475+0000 7f814f7fe640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f81440afc50 0x7f81440b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a24697570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f815659c640 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a24697030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f8156d9d640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f81440afc50 0x7f81440b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 -- 172.21.15.112:0/108140010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 msgr2=0x560a24697030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a24697030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 -- 172.21.15.112:0/108140010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 msgr2=0x560a24697570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a24697570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814f7fe640 1 -- 172.21.15.112:0/1309512677 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f81500df7f0 con 0x7f815810e5a0 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 -- 172.21.15.112:0/108140010 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a245c40d0 con 0x560a242f71b0 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814ffff640 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a24696af0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f81400137e0 tx=0x7f8140005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e00bb520c4b2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:01.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f8156d9d640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f81440afc50 0x7f81440b2160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f81500dcfd0 tx=0x7f8150090040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814cff9640 1 -- 172.21.15.112:0/108140010 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140013e20 con 0x560a242f71b0 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814cff9640 1 -- 172.21.15.112:0/108140010 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8140013130 con 0x560a242f71b0 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.476+0000 7f814cff9640 1 -- 172.21.15.112:0/108140010 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81400047a0 con 0x560a242f71b0 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 -- 172.21.15.112:0/108140010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 msgr2=0x560a24696af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a24696af0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f81400137e0 tx=0x7f8140005850 comp rx=0 tx=0).stop 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 -- 172.21.15.112:0/108140010 shutdown_connections 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a24697030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a24697570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 --2- 172.21.15.112:0/108140010 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a24696af0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 -- 172.21.15.112:0/108140010 >> 172.21.15.112:0/108140010 conn(0x560a242eb300 msgr2=0x560a246932a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 -- 172.21.15.112:0/108140010 shutdown_connections 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.477+0000 7f81639ed740 1 -- 172.21.15.112:0/108140010 wait complete. 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 -- start start 2024-04-25T16:45:01.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 Processor -- start 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a246a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a246a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560a246a3bc0 con 0x560a242f71b0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560a24584b60 con 0x560a242d64d0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560a246a3d40 con 0x560a242ce2f0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f81639ed740 10 client.0 _ll_register_callbacks cb 0x560a245a92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f814ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f814ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40364/0 (socket says 172.21.15.112:40364) 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f814ffff640 1 -- 172.21.15.112:0/1164303142 learned_addr learned my addr 172.21.15.112:0/1164303142 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f815659c640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a246a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f814ffff640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a246a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f815659c640 1 -- 172.21.15.112:0/1164303142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 msgr2=0x560a246a2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f815659c640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a246a2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f815659c640 1 -- 172.21.15.112:0/1164303142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 msgr2=0x560a246a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f815659c640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f815659c640 1 -- 172.21.15.112:0/1164303142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a2430f210 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.478+0000 7f814ffff640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f815659c640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a246a30d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f814802a930 tx=0x7f81480316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea74a07f7427d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f813dffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f813dffb640 1 -- 172.21.15.112:0/1164303142 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148035040 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8140013130 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f813dffb640 1 -- 172.21.15.112:0/1164303142 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f8148004240 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f813dffb640 1 -- 172.21.15.112:0/1164303142 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8148031900 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 10 client.25619 fetch_fsmap learned FSMap version 23 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560a243db620 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f813dffb640 1 -- 172.21.15.112:0/1164303142 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8140013130 con 0x560a242d64d0 2024-04-25T16:45:01.486 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f813dffb640 1 -- 172.21.15.112:0/1164303142 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f81480049e0 con 0x560a242d64d0 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 10 client.25619 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 20 client.25619 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a242b5d80 0x560a246b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 4 client.25619 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.479+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a2460b070 con 0x560a242b5d80 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.480+0000 7f814ffff640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a242b5d80 0x560a246b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.480+0000 7f813dffb640 10 client.25619 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.480+0000 7f814ffff640 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a242b5d80 0x560a246b3710 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d246f9ff1ab8dd server_cookie=bda32d59d162f9ba in_seq=0 out_seq=0 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f813dffb640 1 -- 172.21.15.112:0/1164303142 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560a2460b070 con 0x560a242b5d80 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f813dffb640 10 client.25619 handle_command_reply: tid=0 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 client.25619 shutdown 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 msgr2=0x560a246a30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a246a30d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f814802a930 tx=0x7f81480316d0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.487 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 shutdown_connections 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242d64d0 0x560a246a30d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a242b5d80 0x560a246b3710 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f813dffb640 0 client.25619 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 --2- 172.21.15.112:0/1164303142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242ce2f0 0x560a246a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 >> 172.21.15.112:0/1164303142 conn(0x560a242eb300 msgr2=0x560a246968c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 shutdown_connections 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 1 -- 172.21.15.112:0/1164303142 wait complete. 2024-04-25T16:45:01.488 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.481+0000 7f81639ed740 20 client.25619 trim_cache size 0 max 16384 2024-04-25T16:45:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 Processor -- start 2024-04-25T16:45:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 -- start start 2024-04-25T16:45:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a24693370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242d64d0 0x560a246b8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81581b2340 con 0x560a242f71b0 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81581afc10 con 0x560a242ce2f0 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f81639ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81581ada60 con 0x560a242d64d0 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f814ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f814ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246b9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40380/0 (socket says 172.21.15.112:40380) 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.497+0000 7f814ffff640 1 -- 172.21.15.112:0/3026920488 learned_addr learned my addr 172.21.15.112:0/3026920488 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a24693370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f815659c640 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242d64d0 0x560a246b8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 -- 172.21.15.112:0/3026920488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242d64d0 msgr2=0x560a246b8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242d64d0 0x560a246b8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 -- 172.21.15.112:0/3026920488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 msgr2=0x560a24693370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a24693370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 -- 172.21.15.112:0/3026920488 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a2469f590 con 0x560a242f71b0 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f814ffff640 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246b9400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8140009b90 tx=0x7f8140015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681e81b73533d6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f813ffff640 1 -- 172.21.15.112:0/3026920488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140005180 con 0x560a242f71b0 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f813ffff640 1 -- 172.21.15.112:0/3026920488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814000fc40 con 0x560a242f71b0 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.498+0000 7f813ffff640 1 -- 172.21.15.112:0/3026920488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140030860 con 0x560a242f71b0 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 -- 172.21.15.112:0/3026920488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 msgr2=0x560a246b9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246b9400 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8140009b90 tx=0x7f8140015040 comp rx=0 tx=0).stop 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 -- 172.21.15.112:0/3026920488 shutdown_connections 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242d64d0 0x560a246b8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.504 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242f71b0 0x560a246b9400 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 --2- 172.21.15.112:0/3026920488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a24693370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 -- 172.21.15.112:0/3026920488 >> 172.21.15.112:0/3026920488 conn(0x560a242eb300 msgr2=0x560a24692c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 -- 172.21.15.112:0/3026920488 shutdown_connections 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 -- 172.21.15.112:0/3026920488 wait complete. 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 -- start start 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.499+0000 7f81639ed740 1 Processor -- start 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a246a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 0x560a246a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242f71b0 0x560a24713ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f81581ae0d0 con 0x560a242d64d0 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81581af540 con 0x560a242ce2f0 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81581ae7a0 con 0x560a242f71b0 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 10 client.0 _ll_register_callbacks cb 0x7f8148009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 0x560a246a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f814ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a246a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f814ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a246a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57912/0 (socket says 172.21.15.112:57912) 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f814ffff640 1 -- 172.21.15.112:0/1416809648 learned_addr learned my addr 172.21.15.112:0/1416809648 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 0x560a246a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40386/0 (socket says 172.21.15.112:40386) 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f814ffff640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242f71b0 0x560a24713ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 -- 172.21.15.112:0/1416809648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242f71b0 msgr2=0x560a24713ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242f71b0 0x560a24713ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 -- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 msgr2=0x560a246a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a246a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 -- 172.21.15.112:0/1416809648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a24713ff0 con 0x560a242d64d0 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f815659c640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 0x560a246a69c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8148004db0 tx=0x7f814802ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d0bbbd8c69c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f813dffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f813dffb640 1 -- 172.21.15.112:0/1416809648 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148053040 con 0x560a242d64d0 2024-04-25T16:45:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f813dffb640 1 -- 172.21.15.112:0/1416809648 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8556+0+0 (secure 0 0 0) 0x7f81480049f0 con 0x560a242d64d0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.500+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814000fc40 con 0x560a242d64d0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f813dffb640 1 -- 172.21.15.112:0/1416809648 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8148035040 con 0x560a242d64d0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 10 client.15927 fetch_fsmap learned FSMap version 23 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560a2471b2f0 con 0x560a242d64d0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f813dffb640 1 -- 172.21.15.112:0/1416809648 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814000fc40 con 0x560a242d64d0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f813dffb640 1 -- 172.21.15.112:0/1416809648 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f81480035f0 con 0x560a242d64d0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 10 client.15927 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 20 client.15927 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a2428b1c0 0x560a2471da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 4 client.15927 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a24604ee0 con 0x560a2428b1c0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.501+0000 7f814ffff640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a2428b1c0 0x560a2471da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:01.507 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.502+0000 7f813dffb640 10 client.15927 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.502+0000 7f814ffff640 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a2428b1c0 0x560a2471da50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=149bd190b1481db5 server_cookie=9f491e1075879133 in_seq=0 out_seq=0 2024-04-25T16:45:01.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:01 smithi112 ceph-mon[29122]: pgmap v200: 97 pgs: 97 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 387 op/s 2024-04-25T16:45:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f813dffb640 1 -- 172.21.15.112:0/1416809648 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a24604ee0 con 0x560a2428b1c0 2024-04-25T16:45:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f813dffb640 10 client.15927 handle_command_reply: tid=0 2024-04-25T16:45:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 client.15927 shutdown 2024-04-25T16:45:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 msgr2=0x560a246a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 0x560a246a69c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8148004db0 tx=0x7f814802ab70 comp rx=0 tx=0).stop 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 shutdown_connections 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560a2428b1c0 0x560a2471da50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560a242f71b0 0x560a24713ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560a242d64d0 0x560a246a69c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f813dffb640 0 client.15927 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 --2- 172.21.15.112:0/1416809648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560a242ce2f0 0x560a246a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 >> 172.21.15.112:0/1416809648 conn(0x560a242eb300 msgr2=0x560a246a26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 shutdown_connections 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 1 -- 172.21.15.112:0/1416809648 wait complete. 2024-04-25T16:45:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.687+0000 7f81639ed740 20 client.15927 trim_cache size 0 max 16384 2024-04-25T16:45:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:45:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:45:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:45:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:45:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f81440afc50 msgr2=0x7f81440b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f81440afc50 0x7f81440b2160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f81500dcfd0 tx=0x7f8150090040 comp rx=0 tx=0).stop 2024-04-25T16:45:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 msgr2=0x7f815819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 0x7f815819efc0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f815000eb40 tx=0x7f815000c2d0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 shutdown_connections 2024-04-25T16:45:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f81440afc50 0x7f81440b2160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f815810e5a0 0x7f815819efc0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8158109bf0 0x7f815819ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 --2- 172.21.15.112:0/1309512677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81581020a0 0x7f815819a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 >> 172.21.15.112:0/1309512677 conn(0x7f81580778b0 msgr2=0x7f815810a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 shutdown_connections 2024-04-25T16:45:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:01.688+0000 7f815ca61640 1 -- 172.21.15.112:0/1309512677 wait complete. 2024-04-25T16:45:01.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:01 smithi133 ceph-mon[32691]: pgmap v200: 97 pgs: 97 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 387 op/s 2024-04-25T16:45:01.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:01 smithi116 ceph-mon[32712]: pgmap v200: 97 pgs: 97 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 387 op/s 2024-04-25T16:45:01.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 685680 11.25 MB/sec execute 187 sec latency 87.085 ms 2024-04-25T16:45:02.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 687104 11.22 MB/sec execute 188 sec latency 69.519 ms 2024-04-25T16:45:03.559 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:03 smithi116 ceph-mon[32712]: pgmap v201: 97 pgs: 97 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-25T16:45:03.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:03 smithi112 ceph-mon[29122]: pgmap v201: 97 pgs: 97 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-25T16:45:03.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:03 smithi133 ceph-mon[32691]: pgmap v201: 97 pgs: 97 active+clean; 80 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-25T16:45:03.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 689347 11.23 MB/sec execute 189 sec latency 18.825 ms 2024-04-25T16:45:04.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 691366 11.21 MB/sec execute 190 sec latency 16.781 ms 2024-04-25T16:45:05.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:05 smithi112 ceph-mon[29122]: pgmap v202: 97 pgs: 97 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-04-25T16:45:05.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:05 smithi133 ceph-mon[32691]: pgmap v202: 97 pgs: 97 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-04-25T16:45:05.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:05 smithi116 ceph-mon[32712]: pgmap v202: 97 pgs: 97 active+clean; 69 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-04-25T16:45:05.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 693378 11.22 MB/sec execute 191 sec latency 16.404 ms 2024-04-25T16:45:06.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 694860 11.19 MB/sec execute 192 sec latency 49.857 ms 2024-04-25T16:45:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:07 smithi112 ceph-mon[29122]: pgmap v203: 97 pgs: 97 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-04-25T16:45:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:07 smithi112 ceph-mon[29122]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T16:45:07.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:07 smithi133 ceph-mon[32691]: pgmap v203: 97 pgs: 97 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-04-25T16:45:07.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:07 smithi133 ceph-mon[32691]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T16:45:07.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:07 smithi116 ceph-mon[32712]: pgmap v203: 97 pgs: 97 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 357 op/s 2024-04-25T16:45:07.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:07 smithi116 ceph-mon[32712]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T16:45:07.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 695861 11.17 MB/sec execute 193 sec latency 89.728 ms 2024-04-25T16:45:08.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 697425 11.17 MB/sec execute 194 sec latency 17.940 ms 2024-04-25T16:45:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:09 smithi112 ceph-mon[29122]: pgmap v205: 97 pgs: 97 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-04-25T16:45:09.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:09 smithi133 ceph-mon[32691]: pgmap v205: 97 pgs: 97 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-04-25T16:45:09.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:09 smithi116 ceph-mon[32712]: pgmap v205: 97 pgs: 97 active+clean; 69 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-04-25T16:45:09.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 698622 11.12 MB/sec execute 195 sec latency 19.630 ms 2024-04-25T16:45:10.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 700561 11.13 MB/sec execute 196 sec latency 13.909 ms 2024-04-25T16:45:11.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:11 smithi112 ceph-mon[29122]: pgmap v206: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:45:11.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:11 smithi133 ceph-mon[32691]: pgmap v206: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:45:11.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:11 smithi116 ceph-mon[32712]: pgmap v206: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:45:11.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 703092 11.15 MB/sec execute 197 sec latency 17.055 ms 2024-04-25T16:45:12.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 704405 11.12 MB/sec execute 198 sec latency 96.168 ms 2024-04-25T16:45:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:13 smithi112 ceph-mon[29122]: pgmap v207: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:45:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:13 smithi112 ceph-mon[29122]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T16:45:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:13 smithi133 ceph-mon[32691]: pgmap v207: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:45:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:13 smithi133 ceph-mon[32691]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T16:45:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:13 smithi116 ceph-mon[32712]: pgmap v207: 97 pgs: 97 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:45:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:13 smithi116 ceph-mon[32712]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T16:45:13.881 INFO:tasks.workunit.client.0.smithi112.stdout: 1 706686 11.14 MB/sec execute 199 sec latency 21.688 ms 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 -- 172.21.15.112:0/2523850705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108f00 msgr2=0x7f83f0102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 --2- 172.21.15.112:0/2523850705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108f00 0x7f83f0102b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009900 tx=0x7f83e002a410 comp rx=0 tx=0).stop 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 -- 172.21.15.112:0/2523850705 shutdown_connections 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 --2- 172.21.15.112:0/2523850705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f01031c0 0x7f83f010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 --2- 172.21.15.112:0/2523850705 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108f00 0x7f83f0102b50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 --2- 172.21.15.112:0/2523850705 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f0108a60 0x7f83f0102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 -- 172.21.15.112:0/2523850705 >> 172.21.15.112:0/2523850705 conn(0x7f83f00781b0 msgr2=0x7f83f00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 -- 172.21.15.112:0/2523850705 shutdown_connections 2024-04-25T16:45:14.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.788+0000 7f83f7842640 1 -- 172.21.15.112:0/2523850705 wait complete. 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 Processor -- start 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 -- start start 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f01031c0 0x7f83f01a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 0x7f83f01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f0108f00 0x7f83f019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0113d90 con 0x7f83f0108a60 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0113c10 con 0x7f83f0108f00 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f7842640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0113f10 con 0x7f83f01031c0 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 0x7f83f01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f55de640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f01031c0 0x7f83f01a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 0x7f83f01a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39104/0 (socket says 172.21.15.112:39104) 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f4ddd640 1 -- 172.21.15.112:0/3891300972 learned_addr learned my addr 172.21.15.112:0/3891300972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f0108f00 0x7f83f019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f55de640 1 -- 172.21.15.112:0/3891300972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f0108f00 msgr2=0x7f83f019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f55de640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f0108f00 0x7f83f019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f55de640 1 -- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 msgr2=0x7f83f01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f55de640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 0x7f83f01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f55de640 1 -- 172.21.15.112:0/3891300972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f019ad80 con 0x7f83f01031c0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f0108f00 0x7f83f019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.789+0000 7f83f4ddd640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 0x7f83f01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.790+0000 7f83f55de640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f01031c0 0x7f83f01a04d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83f0103750 tx=0x7f83ec00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.790+0000 7f83e67fc640 1 -- 172.21.15.112:0/3891300972 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00cc40 con 0x7f83f01031c0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.790+0000 7f83e67fc640 1 -- 172.21.15.112:0/3891300972 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec00cde0 con 0x7f83f01031c0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.790+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f019b010 con 0x7f83f01031c0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.790+0000 7f83e67fc640 1 -- 172.21.15.112:0/3891300972 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec004a00 con 0x7f83f01031c0 2024-04-25T16:45:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.790+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f01a7270 con 0x7f83f01031c0 2024-04-25T16:45:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.792+0000 7f83e67fc640 1 -- 172.21.15.112:0/3891300972 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f83ec004ba0 con 0x7f83f01031c0 2024-04-25T16:45:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 Processor -- start 2024-04-25T16:45:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 -- start start 2024-04-25T16:45:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a6aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a6b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a8e39d1460 con 0x55a8e36aa4d0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a8e398b590 con 0x55a8e36a22f0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a8e39cd420 con 0x55a8e36cb1b0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a6b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:41736/0 (socket says 172.21.15.112:41736) 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 -- 172.21.15.112:0/915273299 learned_addr learned my addr 172.21.15.112:0/915273299 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83e67fc640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f83dc0afc50 0x7f83dc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a6aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f83dc0afc50 0x7f83dc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 -- 172.21.15.112:0/915273299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 msgr2=0x55a8e3a6aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a6aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 -- 172.21.15.112:0/915273299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36aa4d0 msgr2=0x55a8e3a6b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a6b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f55de640 1 -- 172.21.15.112:0/915273299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e39980d0 con 0x55a8e36a22f0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83f4ddd640 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.793+0000 7f83e67fc640 1 -- 172.21.15.112:0/3891300972 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8600+0+0 (secure 0 0 0) 0x7f83ec0df030 con 0x7f83f01031c0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.794+0000 7f83f55de640 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6b030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0a6e80 tx=0x7f83ec00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79421343dcf72248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.794+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f83dc0afc50 0x7f83dc0b2160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55a8e3a67c60 tx=0x7f83e8004ab0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.794+0000 7f83db7fe640 1 -- 172.21.15.112:0/915273299 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec01c070 con 0x55a8e36a22f0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.794+0000 7f83db7fe640 1 -- 172.21.15.112:0/915273299 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec0a6c90 con 0x55a8e36a22f0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.794+0000 7f83db7fe640 1 -- 172.21.15.112:0/915273299 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0e2690 con 0x55a8e36a22f0 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 -- 172.21.15.112:0/915273299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 msgr2=0x55a8e3a6b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6b030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0a6e80 tx=0x7f83ec00cf90 comp rx=0 tx=0).stop 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 -- 172.21.15.112:0/915273299 shutdown_connections 2024-04-25T16:45:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a6b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6b030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/915273299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a6aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 -- 172.21.15.112:0/915273299 >> 172.21.15.112:0/915273299 conn(0x55a8e36bf300 msgr2=0x55a8e3a672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 -- 172.21.15.112:0/915273299 shutdown_connections 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 -- 172.21.15.112:0/915273299 wait complete. 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 -- start start 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.795+0000 7f83fe7cb740 1 Processor -- start 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a76b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a75010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3a77bc0 con 0x55a8e36a22f0 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3958b60 con 0x55a8e36aa4d0 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a8e3a77d40 con 0x55a8e36cb1b0 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83fe7cb740 10 client.0 _ll_register_callbacks cb 0x55a8e397d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f55de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a76b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a770b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:41752/0 (socket says 172.21.15.112:41752) 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f5ddf640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a75010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 -- 172.21.15.112:0/3841309060 learned_addr learned my addr 172.21.15.112:0/3841309060 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 -- 172.21.15.112:0/3841309060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 msgr2=0x55a8e3a75010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a75010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 -- 172.21.15.112:0/3841309060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 msgr2=0x55a8e3a76b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a76b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 -- 172.21.15.112:0/3841309060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e36e3210 con 0x55a8e36aa4d0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f55de640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a76b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83f4ddd640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a770b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83e0006010 tx=0x7f83e0031740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e94f736599080d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.796+0000 7f83d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83d8ff9640 1 -- 172.21.15.112:0/3841309060 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0035040 con 0x55a8e36aa4d0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83d8ff9640 1 -- 172.21.15.112:0/3841309060 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8600+0+0 (secure 0 0 0) 0x7f83e0004240 con 0x55a8e36aa4d0 2024-04-25T16:45:14.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec0a6c90 con 0x55a8e36aa4d0 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83d8ff9640 1 -- 172.21.15.112:0/3841309060 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f83e0004440 con 0x55a8e36aa4d0 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 10 client.25637 fetch_fsmap learned FSMap version 23 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a8e37af620 con 0x55a8e36aa4d0 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83d8ff9640 1 -- 172.21.15.112:0/3841309060 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec0a6c90 con 0x55a8e36aa4d0 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83d8ff9640 1 -- 172.21.15.112:0/3841309060 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f83e0048780 con 0x55a8e36aa4d0 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 10 client.25637 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 20 client.25637 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e3689d80 0x55a8e3a87740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 4 client.25637 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e39df070 con 0x55a8e3689d80 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.797+0000 7f83f55de640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e3689d80 0x55a8e3a87740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.798+0000 7f83d8ff9640 10 client.25637 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:14.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.798+0000 7f83f55de640 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e3689d80 0x55a8e3a87740 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219e7913cfc49feb server_cookie=8e524878bee0f9dc in_seq=0 out_seq=0 2024-04-25T16:45:14.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83d8ff9640 1 -- 172.21.15.112:0/3841309060 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a8e39df070 con 0x55a8e3689d80 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83d8ff9640 10 client.25637 handle_command_reply: tid=0 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 client.25637 shutdown 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 msgr2=0x55a8e3a770b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a770b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83e0006010 tx=0x7f83e0031740 comp rx=0 tx=0).stop 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 shutdown_connections 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e3689d80 0x55a8e3a87740 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a75010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83d8ff9640 0 client.25637 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a770b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/3841309060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a76b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 >> 172.21.15.112:0/3841309060 conn(0x55a8e36bf300 msgr2=0x55a8e3a6a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 shutdown_connections 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.799+0000 7f83fe7cb740 1 -- 172.21.15.112:0/3841309060 wait complete. 2024-04-25T16:45:14.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.800+0000 7f83fe7cb740 20 client.25637 trim_cache size 0 max 16384 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 Processor -- start 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 -- start start 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a70810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b2420 con 0x55a8e36a22f0 2024-04-25T16:45:14.814 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01afcf0 con 0x55a8e36cb1b0 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01adb60 con 0x55a8e36aa4d0 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f5ddf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39122/0 (socket says 172.21.15.112:39122) 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a70810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f5ddf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:41764/0 (socket says 172.21.15.112:41764) 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 -- 172.21.15.112:0/1742613476 learned_addr learned my addr 172.21.15.112:0/1742613476 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 -- 172.21.15.112:0/1742613476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36aa4d0 msgr2=0x55a8e3a70810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a70810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 -- 172.21.15.112:0/1742613476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 msgr2=0x55a8e3a7cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.815+0000 7f83f55de640 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.816+0000 7f83f55de640 1 -- 172.21.15.112:0/1742613476 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e3a75570 con 0x55a8e36a22f0 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.816+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.815 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.816+0000 7f83f55de640 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6eec0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f83ec004610 tx=0x7f83ec0a30c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c18463f2afc5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.816+0000 7f83daffd640 1 -- 172.21.15.112:0/1742613476 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec01c040 con 0x55a8e36a22f0 2024-04-25T16:45:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.816+0000 7f83daffd640 1 -- 172.21.15.112:0/1742613476 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec0a6c90 con 0x55a8e36a22f0 2024-04-25T16:45:14.816 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.816+0000 7f83daffd640 1 -- 172.21.15.112:0/1742613476 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0affe0 con 0x55a8e36a22f0 2024-04-25T16:45:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1742613476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 msgr2=0x55a8e3a6eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6eec0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f83ec004610 tx=0x7f83ec0a30c0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1742613476 shutdown_connections 2024-04-25T16:45:14.819 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36aa4d0 0x55a8e3a70810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1742613476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36a22f0 0x55a8e3a6eec0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1742613476 >> 172.21.15.112:0/1742613476 conn(0x55a8e36bf300 msgr2=0x55a8e3a6e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1742613476 shutdown_connections 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1742613476 wait complete. 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 -- start start 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.817+0000 7f83fe7cb740 1 Processor -- start 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36a22f0 0x55a8e3ae8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3ae8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01ae1b0 con 0x55a8e36cb1b0 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01af620 con 0x55a8e36aa4d0 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01ae880 con 0x55a8e36a22f0 2024-04-25T16:45:14.820 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 10 client.0 _ll_register_callbacks cb 0x7f83e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39132/0 (socket says 172.21.15.112:39132) 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f4ddd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3ae8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 -- 172.21.15.112:0/1514297135 learned_addr learned my addr 172.21.15.112:0/1514297135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f55de640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36a22f0 0x55a8e3ae8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 -- 172.21.15.112:0/1514297135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36a22f0 msgr2=0x55a8e3ae8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36a22f0 0x55a8e3ae8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 -- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 msgr2=0x55a8e3ae8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3ae8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 -- 172.21.15.112:0/1514297135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e3a7cdd0 con 0x55a8e36cb1b0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f4ddd640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3ae8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83f5ddf640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7c7c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f83e8010e70 tx=0x7f83e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d8462228696ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83d8ff9640 1 -- 172.21.15.112:0/1514297135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800fd40 con 0x55a8e36cb1b0 2024-04-25T16:45:14.821 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83d8ff9640 1 -- 172.21.15.112:0/1514297135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8600+0+0 (secure 0 0 0) 0x7f83e8004b20 con 0x55a8e36cb1b0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.818+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec0a6c90 con 0x55a8e36cb1b0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83d8ff9640 1 -- 172.21.15.112:0/1514297135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f83e8004d20 con 0x55a8e36cb1b0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 10 client.15945 fetch_fsmap learned FSMap version 23 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a8e3aef8c0 con 0x55a8e36cb1b0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83d8ff9640 1 -- 172.21.15.112:0/1514297135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83ec0a6c90 con 0x55a8e36cb1b0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83d8ff9640 1 -- 172.21.15.112:0/1514297135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f83e801f040 con 0x55a8e36cb1b0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 10 client.15945 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 20 client.15945 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e365f1c0 0x55a8e3af2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 4 client.15945 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a8e39d8ee0 con 0x55a8e365f1c0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.819+0000 7f83f55de640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e365f1c0 0x55a8e3af2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.820+0000 7f83d8ff9640 10 client.15945 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:14.822 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.820+0000 7f83f55de640 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e365f1c0 0x55a8e3af2100 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a74af8d339029ec8 server_cookie=28a0b14bf92dd0f1 in_seq=0 out_seq=0 2024-04-25T16:45:14.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 708179 11.12 MB/sec execute 200 sec latency 32.340 ms 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.961+0000 7f83d8ff9640 1 -- 172.21.15.112:0/1514297135 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a8e39d8ee0 con 0x55a8e365f1c0 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.961+0000 7f83d8ff9640 10 client.15945 handle_command_reply: tid=0 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.961+0000 7f83fe7cb740 1 client.15945 shutdown 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 msgr2=0x55a8e3a7c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7c7c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f83e8010e70 tx=0x7f83e8004a40 comp rx=0 tx=0).stop 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 shutdown_connections 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a8e36cb1b0 0x55a8e3a7c7c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a8e36aa4d0 0x55a8e3ae8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a8e365f1c0 0x55a8e3af2100 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 --2- 172.21.15.112:0/1514297135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a8e36a22f0 0x55a8e3ae8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83d8ff9640 0 client.15945 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 >> 172.21.15.112:0/1514297135 conn(0x55a8e36bf300 msgr2=0x55a8e3a6a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 shutdown_connections 2024-04-25T16:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 1 -- 172.21.15.112:0/1514297135 wait complete. 2024-04-25T16:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.962+0000 7f83fe7cb740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T16:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:45:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f83dc0afc50 msgr2=0x7f83dc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f83dc0afc50 0x7f83dc0b2160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55a8e3a67c60 tx=0x7f83e8004ab0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f01031c0 msgr2=0x7f83f01a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f01031c0 0x7f83f01a04d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83f0103750 tx=0x7f83ec00ee50 comp rx=0 tx=0).stop 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 shutdown_connections 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f83dc0afc50 0x7f83dc0b2160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f0108f00 0x7f83f019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f83f0108a60 0x7f83f01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 --2- 172.21.15.112:0/3891300972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f01031c0 0x7f83f01a04d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:14.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 >> 172.21.15.112:0/3891300972 conn(0x7f83f00781b0 msgr2=0x7f83f0104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 shutdown_connections 2024-04-25T16:45:14.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:14.963+0000 7f83f7842640 1 -- 172.21.15.112:0/3891300972 wait complete. 2024-04-25T16:45:15.321 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:15 smithi116 ceph-mon[32712]: pgmap v209: 97 pgs: 97 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2024-04-25T16:45:15.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:15 smithi112 ceph-mon[29122]: pgmap v209: 97 pgs: 97 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2024-04-25T16:45:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:15 smithi133 ceph-mon[32691]: pgmap v209: 97 pgs: 97 active+clean; 79 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 207 op/s 2024-04-25T16:45:15.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:45:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setuser ceph since I am not root 2024-04-25T16:45:15.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:45:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setgroup ceph since I am not root 2024-04-25T16:45:15.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:45:15 smithi116 ceph-mds[62762]: -- 172.21.15.116:0/1395706561 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642e0815a40 con 0x5642e085d180 2024-04-25T16:45:15.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:45:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: starting mds.e at 2024-04-25T16:45:15.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 710481 11.13 MB/sec execute 201 sec latency 110.829 ms 2024-04-25T16:45:16.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:16 smithi112 ceph-mon[29122]: daemon mds.e restarted 2024-04-25T16:45:16.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:16 smithi133 ceph-mon[32691]: daemon mds.e restarted 2024-04-25T16:45:16.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:16 smithi116 ceph-mon[32712]: daemon mds.e restarted 2024-04-25T16:45:16.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 712085 11.10 MB/sec execute 202 sec latency 18.551 ms 2024-04-25T16:45:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:17 smithi112 ceph-mon[29122]: pgmap v210: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-04-25T16:45:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:17 smithi112 ceph-mon[29122]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T16:45:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:17 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6834/2535093430,v1:172.21.15.116:6835/2535093430] up:boot 2024-04-25T16:45:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:17 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:45:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:17 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:45:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:17 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:45:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:17 smithi133 ceph-mon[32691]: pgmap v210: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-04-25T16:45:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:17 smithi133 ceph-mon[32691]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T16:45:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:17 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6834/2535093430,v1:172.21.15.116:6835/2535093430] up:boot 2024-04-25T16:45:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:17 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:45:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:17 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:45:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:17 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:45:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:17 smithi116 ceph-mon[32712]: pgmap v210: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.8 MiB/s wr, 225 op/s 2024-04-25T16:45:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:17 smithi116 ceph-mon[32712]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T16:45:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:17 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6834/2535093430,v1:172.21.15.116:6835/2535093430] up:boot 2024-04-25T16:45:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:17 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:45:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:17 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:45:17.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:17 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:45:17.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 714131 11.12 MB/sec execute 203 sec latency 48.840 ms 2024-04-25T16:45:18.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 715856 11.09 MB/sec execute 204 sec latency 74.215 ms 2024-04-25T16:45:19.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:19 smithi112 ceph-mon[29122]: pgmap v212: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 141 op/s 2024-04-25T16:45:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:19 smithi133 ceph-mon[32691]: pgmap v212: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 141 op/s 2024-04-25T16:45:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:19 smithi116 ceph-mon[32712]: pgmap v212: 97 pgs: 97 active+clean; 83 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 141 op/s 2024-04-25T16:45:19.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 717901 11.10 MB/sec execute 205 sec latency 12.181 ms 2024-04-25T16:45:20.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 720233 11.11 MB/sec execute 206 sec latency 22.677 ms 2024-04-25T16:45:21.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:21 smithi112 ceph-mon[29122]: pgmap v213: 97 pgs: 97 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-04-25T16:45:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:21 smithi133 ceph-mon[32691]: pgmap v213: 97 pgs: 97 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-04-25T16:45:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:21 smithi116 ceph-mon[32712]: pgmap v213: 97 pgs: 97 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.6 MiB/s wr, 419 op/s 2024-04-25T16:45:21.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 722077 11.10 MB/sec execute 207 sec latency 16.277 ms 2024-04-25T16:45:22.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 724498 11.12 MB/sec execute 208 sec latency 22.674 ms 2024-04-25T16:45:23.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:23 smithi112 ceph-mon[29122]: pgmap v214: 97 pgs: 97 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 351 op/s 2024-04-25T16:45:23.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:23 smithi133 ceph-mon[32691]: pgmap v214: 97 pgs: 97 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 351 op/s 2024-04-25T16:45:23.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:23 smithi116 ceph-mon[32712]: pgmap v214: 97 pgs: 97 active+clean; 70 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 351 op/s 2024-04-25T16:45:23.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 725963 11.10 MB/sec execute 209 sec latency 103.414 ms 2024-04-25T16:45:24.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 728349 11.11 MB/sec execute 210 sec latency 17.340 ms 2024-04-25T16:45:25.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:25 smithi112 ceph-mon[29122]: pgmap v215: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 376 op/s 2024-04-25T16:45:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:25 smithi133 ceph-mon[32691]: pgmap v215: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 376 op/s 2024-04-25T16:45:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:25 smithi116 ceph-mon[32712]: pgmap v215: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 376 op/s 2024-04-25T16:45:25.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 730382 11.09 MB/sec execute 211 sec latency 34.923 ms 2024-04-25T16:45:26.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 732403 11.10 MB/sec execute 212 sec latency 12.886 ms 2024-04-25T16:45:27.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:27 smithi112 ceph-mon[29122]: pgmap v216: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2024-04-25T16:45:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:27 smithi133 ceph-mon[32691]: pgmap v216: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2024-04-25T16:45:27.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:27 smithi116 ceph-mon[32712]: pgmap v216: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 339 op/s 2024-04-25T16:45:27.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 735018 11.12 MB/sec execute 213 sec latency 22.562 ms 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 -- 172.21.15.112:0/754491017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108f00 msgr2=0x7f5a2c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/754491017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108f00 0x7f5a2c102b50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009900 tx=0x7f5a1c02a410 comp rx=0 tx=0).stop 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 -- 172.21.15.112:0/754491017 shutdown_connections 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/754491017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c1031c0 0x7f5a2c10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/754491017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108f00 0x7f5a2c102b50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/754491017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c108a60 0x7f5a2c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 -- 172.21.15.112:0/754491017 >> 172.21.15.112:0/754491017 conn(0x7f5a2c0781b0 msgr2=0x7f5a2c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 -- 172.21.15.112:0/754491017 shutdown_connections 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.079+0000 7f5a33a5a640 1 -- 172.21.15.112:0/754491017 wait complete. 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 Processor -- start 2024-04-25T16:45:28.080 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 -- start start 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c1031c0 0x7f5a2c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 0x7f5a2c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c108f00 0x7f5a2c19c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c112c80 con 0x7f5a2c108a60 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c112b00 con 0x7f5a2c1031c0 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a33a5a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c112e00 con 0x7f5a2c108f00 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 0x7f5a2c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a317f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c1031c0 0x7f5a2c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 0x7f5a2c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34292/0 (socket says 172.21.15.112:34292) 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.080+0000 7f5a30ff5640 1 -- 172.21.15.112:0/987652125 learned_addr learned my addr 172.21.15.112:0/987652125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c108f00 0x7f5a2c19c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a30ff5640 1 -- 172.21.15.112:0/987652125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c108f00 msgr2=0x7f5a2c19c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.081 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c108f00 0x7f5a2c19c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a30ff5640 1 -- 172.21.15.112:0/987652125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c1031c0 msgr2=0x7f5a2c19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c1031c0 0x7f5a2c19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a30ff5640 1 -- 172.21.15.112:0/987652125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2c1a68f0 con 0x7f5a2c108a60 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a317f6640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c1031c0 0x7f5a2c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 0x7f5a2c19bb20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006010 tx=0x7f5a1c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a227fc640 1 -- 172.21.15.112:0/987652125 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c03f030 con 0x7f5a2c108a60 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2c1a6b80 con 0x7f5a2c108a60 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a227fc640 1 -- 172.21.15.112:0/987652125 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c003aa0 con 0x7f5a2c108a60 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a227fc640 1 -- 172.21.15.112:0/987652125 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c035070 con 0x7f5a2c108a60 2024-04-25T16:45:28.082 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.081+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2c1a70c0 con 0x7f5a2c108a60 2024-04-25T16:45:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.084+0000 7f5a3a9ea740 1 Processor -- start 2024-04-25T16:45:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.084+0000 7f5a3a9ea740 1 -- start start 2024-04-25T16:45:28.084 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a227fc640 1 -- 172.21.15.112:0/987652125 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f5a1c042030 con 0x7f5a2c108a60 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ccfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6849072f0 0x55d684cd0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d68490f4d0 0x55d684cd0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d684c36460 con 0x55d6849072f0 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d684bf0590 con 0x55d68490f4d0 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d684c32420 con 0x55d6849301b0 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a317f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6849072f0 0x55d684cd0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d68490f4d0 0x55d684cd0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ccfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a317f6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6849072f0 0x55d684cd0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34300/0 (socket says 172.21.15.112:34300) 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a317f6640 1 -- 172.21.15.112:0/1510827372 learned_addr learned my addr 172.21.15.112:0/1510827372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 -- 172.21.15.112:0/1510827372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d68490f4d0 msgr2=0x55d684cd0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.085 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d68490f4d0 0x55d684cd0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 -- 172.21.15.112:0/1510827372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6849072f0 msgr2=0x55d684cd0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6849072f0 0x55d684cd0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 -- 172.21.15.112:0/1510827372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d684bfd0d0 con 0x55d6849301b0 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ccfaf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a24005fd0 tx=0x7f5a24010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0991866a3ea310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.085+0000 7f5a227fc640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5a180afc50 0x7f5a180b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.086+0000 7f5a317f6640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5a180afc50 0x7f5a180b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.086+0000 7f5a227fc640 1 -- 172.21.15.112:0/987652125 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7f5a1c0ff450 con 0x7f5a2c108a60 2024-04-25T16:45:28.086 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.086+0000 7f5a317f6640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5a180afc50 0x7f5a180b2160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d684cd0ea0 tx=0x7f5a28005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.087+0000 7f5a177fe640 1 -- 172.21.15.112:0/1510827372 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24016070 con 0x55d6849301b0 2024-04-25T16:45:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.087+0000 7f5a177fe640 1 -- 172.21.15.112:0/1510827372 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a24004d20 con 0x55d6849301b0 2024-04-25T16:45:28.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.087+0000 7f5a177fe640 1 -- 172.21.15.112:0/1510827372 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a24012810 con 0x55d6849301b0 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1510827372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 msgr2=0x55d684ccfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ccfaf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5a24005fd0 tx=0x7f5a24010740 comp rx=0 tx=0).stop 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1510827372 shutdown_connections 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d68490f4d0 0x55d684cd0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d6849072f0 0x55d684cd0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1510827372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ccfaf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1510827372 >> 172.21.15.112:0/1510827372 conn(0x55d684924300 msgr2=0x55d684ccc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1510827372 shutdown_connections 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.088+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1510827372 wait complete. 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 -- start start 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 Processor -- start 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684cda1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cda6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ce0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55d684cdcc20 con 0x55d68490f4d0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d684bbdb60 con 0x55d6849072f0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d684cdcda0 con 0x55d6849301b0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.089+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cda6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a3a9ea740 10 client.0 _ll_register_callbacks cb 0x55d684be22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684cda1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cda6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34312/0 (socket says 172.21.15.112:34312) 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a30ff5640 1 -- 172.21.15.112:0/4009471143 learned_addr learned my addr 172.21.15.112:0/4009471143 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ce0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 -- 172.21.15.112:0/4009471143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 msgr2=0x55d684ce0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ce0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 -- 172.21.15.112:0/4009471143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 msgr2=0x55d684cda6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cda6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 -- 172.21.15.112:0/4009471143 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d684948210 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a317f6640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684cda1b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55d684ccd230 tx=0x7f5a28004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155b177b495953c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a14ff9640 1 -- 172.21.15.112:0/4009471143 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2802e060 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a24004d20 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a14ff9640 1 -- 172.21.15.112:0/4009471143 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7f5a28018030 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a14ff9640 1 -- 172.21.15.112:0/4009471143 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a2801e5a0 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a3a9ea740 10 client.25661 fetch_fsmap learned FSMap version 25 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.090+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d684a14620 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a14ff9640 1 -- 172.21.15.112:0/4009471143 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a24004d20 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a14ff9640 1 -- 172.21.15.112:0/4009471143 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5a280318c0 con 0x55d6849072f0 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a3a9ea740 10 client.25661 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:45:28.094 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a3a9ea740 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a3a9ea740 20 client.25661 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848eed80 0x55d684cec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a3a9ea740 4 client.25661 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d684c44070 con 0x55d6848eed80 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848eed80 0x55d684cec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.091+0000 7f5a14ff9640 10 client.25661 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848eed80 0x55d684cec7d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c946a60b5825a4 server_cookie=e413d7d2153094ca in_seq=0 out_seq=0 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a14ff9640 1 -- 172.21.15.112:0/4009471143 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d684c44070 con 0x55d6848eed80 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a14ff9640 10 client.25661 handle_command_reply: tid=0 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 client.25661 shutdown 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 msgr2=0x55d684cda1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684cda1b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55d684ccd230 tx=0x7f5a28004730 comp rx=0 tx=0).stop 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 shutdown_connections 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848eed80 0x55d684cec7d0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684ce0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cda6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/4009471143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684cda1b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 >> 172.21.15.112:0/4009471143 conn(0x55d684924300 msgr2=0x55d684ccf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.092+0000 7f5a14ff9640 0 client.25661 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.093+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 shutdown_connections 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.093+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/4009471143 wait complete. 2024-04-25T16:45:28.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.093+0000 7f5a3a9ea740 20 client.25661 trim_cache size 0 max 16384 2024-04-25T16:45:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 Processor -- start 2024-04-25T16:45:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 -- start start 2024-04-25T16:45:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849072f0 0x55d684cd8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cd3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849301b0 0x55d684ce0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1b23d0 con 0x55d68490f4d0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1afca0 con 0x55d6849301b0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1adaf0 con 0x55d6849072f0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cd3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cd3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:34316/0 (socket says 172.21.15.112:34316) 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a30ff5640 1 -- 172.21.15.112:0/1655579495 learned_addr learned my addr 172.21.15.112:0/1655579495 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849301b0 0x55d684ce0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.108+0000 7f5a317f6640 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849072f0 0x55d684cd8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a30ff5640 1 -- 172.21.15.112:0/1655579495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849072f0 msgr2=0x55d684cd8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849072f0 0x55d684cd8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a30ff5640 1 -- 172.21.15.112:0/1655579495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849301b0 msgr2=0x55d684ce0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849301b0 0x55d684ce0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a30ff5640 1 -- 172.21.15.112:0/1655579495 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d684cd0ed0 con 0x55d68490f4d0 2024-04-25T16:45:28.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cd3e50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0fcb20 tx=0x7f5a1c1019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1335bc6b0cf38d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a16ffd640 1 -- 172.21.15.112:0/1655579495 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c049040 con 0x55d68490f4d0 2024-04-25T16:45:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a16ffd640 1 -- 172.21.15.112:0/1655579495 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c0c7190 con 0x55d68490f4d0 2024-04-25T16:45:28.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.109+0000 7f5a16ffd640 1 -- 172.21.15.112:0/1655579495 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0d0430 con 0x55d68490f4d0 2024-04-25T16:45:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1655579495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 msgr2=0x55d684cd3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cd3e50 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0fcb20 tx=0x7f5a1c1019d0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1655579495 shutdown_connections 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849301b0 0x55d684ce0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684cd3e50 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/1655579495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849072f0 0x55d684cd8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1655579495 >> 172.21.15.112:0/1655579495 conn(0x55d684924300 msgr2=0x55d684cd3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1655579495 shutdown_connections 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/1655579495 wait complete. 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 -- start start 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.110+0000 7f5a3a9ea740 1 Processor -- start 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684ce15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684ce1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684d52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1ae160 con 0x55d68490f4d0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1af5d0 con 0x55d6849072f0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a2c1ae830 con 0x55d6849301b0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a3a9ea740 10 client.0 _ll_register_callbacks cb 0x7f5a28009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684ce15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a30ff5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684ce1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684ce15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:60146/0 (socket says 172.21.15.112:60146) 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 -- 172.21.15.112:0/3113267499 learned_addr learned my addr 172.21.15.112:0/3113267499 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a31ff7640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684d52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 -- 172.21.15.112:0/3113267499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 msgr2=0x55d684d52b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684d52b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 -- 172.21.15.112:0/3113267499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 msgr2=0x55d684ce1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684ce1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 -- 172.21.15.112:0/3113267499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d684ce23b0 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684ce1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a317f6640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684ce15f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a280092e0 tx=0x7f5a28031a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915cb007f2e15af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.111+0000 7f5a14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a14ff9640 1 -- 172.21.15.112:0/3113267499 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28040060 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a14ff9640 1 -- 172.21.15.112:0/3113267499 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7f5a2801e950 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c0c7190 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a14ff9640 1 -- 172.21.15.112:0/3113267499 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a2801eb50 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 10 client.25667 fetch_fsmap learned FSMap version 25 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d684d54100 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a14ff9640 1 -- 172.21.15.112:0/3113267499 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c0c7190 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a14ff9640 1 -- 172.21.15.112:0/3113267499 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5a2803bc30 con 0x55d6849072f0 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 10 client.25667 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:45:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 20 client.25667 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.112+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848c41c0 0x55d684d568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.113+0000 7f5a3a9ea740 4 client.25667 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.113+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d684cd5b80 con 0x55d6848c41c0 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.113+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848c41c0 0x55d684d568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.113+0000 7f5a14ff9640 10 client.25667 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.113+0000 7f5a30ff5640 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848c41c0 0x55d684d568c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6decf9420da03e server_cookie=c9214928ac123e01 in_seq=0 out_seq=0 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.272+0000 7f5a14ff9640 1 -- 172.21.15.112:0/3113267499 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d684cd5b80 con 0x55d6848c41c0 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.272+0000 7f5a14ff9640 10 client.25667 handle_command_reply: tid=0 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.272+0000 7f5a3a9ea740 1 client.25667 shutdown 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.272+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 msgr2=0x55d684ce15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.272+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684ce15f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a280092e0 tx=0x7f5a28031a40 comp rx=0 tx=0).stop 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 shutdown_connections 2024-04-25T16:45:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55d6848c41c0 0x55d684d568c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d6849301b0 0x55d684d52b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55d68490f4d0 0x55d684ce1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 --2- 172.21.15.112:0/3113267499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d6849072f0 0x55d684ce15f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 >> 172.21.15.112:0/3113267499 conn(0x55d684924300 msgr2=0x55d684cdb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a14ff9640 0 client.25667 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 shutdown_connections 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 1 -- 172.21.15.112:0/3113267499 wait complete. 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.273+0000 7f5a3a9ea740 20 client.25667 trim_cache size 0 max 16384 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:45:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:45:28.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5a180afc50 msgr2=0x7f5a180b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5a180afc50 0x7f5a180b2160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d684cd0ea0 tx=0x7f5a28005f90 comp rx=0 tx=0).stop 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 msgr2=0x7f5a2c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 0x7f5a2c19bb20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c006010 tx=0x7f5a1c004060 comp rx=0 tx=0).stop 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 shutdown_connections 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5a180afc50 0x7f5a180b2160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a2c108f00 0x7f5a2c19c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a2c108a60 0x7f5a2c19bb20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 --2- 172.21.15.112:0/987652125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a2c1031c0 0x7f5a2c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 >> 172.21.15.112:0/987652125 conn(0x7f5a2c0781b0 msgr2=0x7f5a2c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 shutdown_connections 2024-04-25T16:45:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:28.274+0000 7f5a33a5a640 1 -- 172.21.15.112:0/987652125 wait complete. 2024-04-25T16:45:28.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 735909 11.08 MB/sec execute 214 sec latency 104.957 ms 2024-04-25T16:45:29.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:29 smithi112 ceph-mon[29122]: pgmap v217: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-04-25T16:45:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:29 smithi133 ceph-mon[32691]: pgmap v217: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-04-25T16:45:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:29 smithi116 ceph-mon[32712]: pgmap v217: 97 pgs: 97 active+clean; 77 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 293 op/s 2024-04-25T16:45:29.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 738413 11.10 MB/sec execute 215 sec latency 14.575 ms 2024-04-25T16:45:30.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 739998 11.09 MB/sec execute 216 sec latency 21.391 ms 2024-04-25T16:45:31.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:31 smithi112 ceph-mon[29122]: pgmap v218: 97 pgs: 97 active+clean; 67 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 481 op/s 2024-04-25T16:45:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:31 smithi133 ceph-mon[32691]: pgmap v218: 97 pgs: 97 active+clean; 67 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 481 op/s 2024-04-25T16:45:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:31 smithi116 ceph-mon[32712]: pgmap v218: 97 pgs: 97 active+clean; 67 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 481 op/s 2024-04-25T16:45:31.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 742269 11.10 MB/sec execute 217 sec latency 19.797 ms 2024-04-25T16:45:32.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 743904 11.08 MB/sec execute 218 sec latency 20.659 ms 2024-04-25T16:45:33.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:33 smithi112 ceph-mon[29122]: pgmap v219: 97 pgs: 97 active+clean; 67 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-04-25T16:45:33.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:33 smithi133 ceph-mon[32691]: pgmap v219: 97 pgs: 97 active+clean; 67 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-04-25T16:45:33.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:33 smithi116 ceph-mon[32712]: pgmap v219: 97 pgs: 97 active+clean; 67 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 295 op/s 2024-04-25T16:45:33.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 746019 11.09 MB/sec execute 219 sec latency 92.115 ms 2024-04-25T16:45:34.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 748023 11.07 MB/sec execute 220 sec latency 17.593 ms 2024-04-25T16:45:35.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:35 smithi112 ceph-mon[29122]: pgmap v220: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 384 op/s 2024-04-25T16:45:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:35 smithi133 ceph-mon[32691]: pgmap v220: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 384 op/s 2024-04-25T16:45:35.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:35 smithi116 ceph-mon[32712]: pgmap v220: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 384 op/s 2024-04-25T16:45:35.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 749937 11.08 MB/sec execute 221 sec latency 13.039 ms 2024-04-25T16:45:36.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 752425 11.09 MB/sec execute 222 sec latency 22.749 ms 2024-04-25T16:45:37.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:37 smithi112 ceph-mon[29122]: pgmap v221: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-04-25T16:45:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:37 smithi133 ceph-mon[32691]: pgmap v221: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-04-25T16:45:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:37 smithi116 ceph-mon[32712]: pgmap v221: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-04-25T16:45:37.882 INFO:tasks.workunit.client.0.smithi112.stdout: 1 754221 11.08 MB/sec execute 223 sec latency 14.833 ms 2024-04-25T16:45:38.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 756451 11.09 MB/sec execute 224 sec latency 103.797 ms 2024-04-25T16:45:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:39 smithi133 ceph-mon[32691]: pgmap v222: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 291 op/s 2024-04-25T16:45:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:39 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:45:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:39 smithi116 ceph-mon[32712]: pgmap v222: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 291 op/s 2024-04-25T16:45:39.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:39 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:45:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:39 smithi112 ceph-mon[29122]: pgmap v222: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 291 op/s 2024-04-25T16:45:39.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:39 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:45:39.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 757985 11.07 MB/sec execute 225 sec latency 16.964 ms 2024-04-25T16:45:40.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:40 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:45:40.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:40 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:45:40.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:40 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:45:40.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:40 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:45:40.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:40 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:45:40.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:40 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:45:40.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:40 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:45:40.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:40 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:45:40.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:40 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:45:40.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 760281 11.08 MB/sec execute 226 sec latency 16.669 ms 2024-04-25T16:45:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.375+0000 7fd7e7a30640 1 -- 172.21.15.112:0/760660750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e0104c70 msgr2=0x7fd7e0105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.375+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/760660750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e0104c70 0x7fd7e0105080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009900 tx=0x7fd7dc02a410 comp rx=0 tx=0).stop 2024-04-25T16:45:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 -- 172.21.15.112:0/760660750 shutdown_connections 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/760660750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e010e0c0 0x7fd7e01105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/760660750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010b6d0 0x7fd7e010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/760660750 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e0104c70 0x7fd7e0105080 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 -- 172.21.15.112:0/760660750 >> 172.21.15.112:0/760660750 conn(0x7fd7e00781b0 msgr2=0x7fd7e00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 -- 172.21.15.112:0/760660750 shutdown_connections 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 -- 172.21.15.112:0/760660750 wait complete. 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 Processor -- start 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.376+0000 7fd7e7a30640 1 -- start start 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e7a30640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e0104c70 0x7fd7e019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e7a30640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 0x7fd7e019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e7a30640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010e0c0 0x7fd7e019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e7a30640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0113d20 con 0x7fd7e010b6d0 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e7a30640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0113ba0 con 0x7fd7e010e0c0 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e7a30640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e0113ea0 con 0x7fd7e0104c70 2024-04-25T16:45:41.377 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 0x7fd7e019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 0x7fd7e019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46976/0 (socket says 172.21.15.112:46976) 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/1756534462 learned_addr learned my addr 172.21.15.112:0/1756534462 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010e0c0 0x7fd7e019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e0104c70 0x7fd7e019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/1756534462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e0104c70 msgr2=0x7fd7e019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e0104c70 0x7fd7e019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/1756534462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010e0c0 msgr2=0x7fd7e019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010e0c0 0x7fd7e019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/1756534462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7e019f560 con 0x7fd7e010b6d0 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010e0c0 0x7fd7e019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 0x7fd7e019ac60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e9f0 tx=0x7fd7d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.377+0000 7fd7d67fc640 1 -- 172.21.15.112:0/1756534462 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d000cd80 con 0x7fd7e010b6d0 2024-04-25T16:45:41.378 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.378+0000 7fd7d67fc640 1 -- 172.21.15.112:0/1756534462 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d00108e0 con 0x7fd7e010b6d0 2024-04-25T16:45:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.378+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7e019f850 con 0x7fd7e010b6d0 2024-04-25T16:45:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.378+0000 7fd7d67fc640 1 -- 172.21.15.112:0/1756534462 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0018980 con 0x7fd7e010b6d0 2024-04-25T16:45:41.379 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.378+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7e01a71d0 con 0x7fd7e010b6d0 2024-04-25T16:45:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.380+0000 7fd7d67fc640 1 -- 172.21.15.112:0/1756534462 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fd7d0010430 con 0x7fd7e010b6d0 2024-04-25T16:45:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.380+0000 7fd7ee9b2740 1 Processor -- start 2024-04-25T16:45:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.380+0000 7fd7ee9b2740 1 -- start start 2024-04-25T16:45:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.380+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.380+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8c8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589ad82e460 con 0x5589ad5281b0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5589ad7e8590 con 0x5589ad5074d0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589ad82a420 con 0x5589ad4ff2f0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7d67fc640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd7cc0afcd0 0x7fd7cc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46982/0 (socket says 172.21.15.112:46982) 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 -- 172.21.15.112:0/93670240 learned_addr learned my addr 172.21.15.112:0/93670240 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd7cc0afcd0 0x7fd7cc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7d67fc640 1 -- 172.21.15.112:0/1756534462 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7fd7d00df3b0 con 0x7fd7e010b6d0 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 -- 172.21.15.112:0/93670240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 msgr2=0x5589ad8c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 -- 172.21.15.112:0/93670240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 msgr2=0x5589ad8c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 -- 172.21.15.112:0/93670240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ad7f50d0 con 0x5589ad5281b0 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8c7af0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8012e60 tx=0x7fd7d800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a311632788c69460 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.381+0000 7fd7cb7fe640 1 -- 172.21.15.112:0/93670240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8012850 con 0x5589ad5281b0 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd7cc0afcd0 0x7fd7cc0b21e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8021000 tx=0x7fd7d801f000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7cb7fe640 1 -- 172.21.15.112:0/93670240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d8022070 con 0x5589ad5281b0 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7cb7fe640 1 -- 172.21.15.112:0/93670240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80046e0 con 0x5589ad5281b0 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/93670240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 msgr2=0x5589ad8c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8c7af0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8012e60 tx=0x7fd7d800e720 comp rx=0 tx=0).stop 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/93670240 shutdown_connections 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.382+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/93670240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8c7af0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/93670240 >> 172.21.15.112:0/93670240 conn(0x5589ad51c300 msgr2=0x5589ad8c42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:41.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/93670240 shutdown_connections 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/93670240 wait complete. 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 -- start start 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 Processor -- start 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.383+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8d4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589ad8d4c20 con 0x5589ad5281b0 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5589ad7b5b60 con 0x5589ad5074d0 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589ad8d4da0 con 0x5589ad4ff2f0 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7ee9b2740 10 client.0 _ll_register_callbacks cb 0x5589ad7da2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e5fcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e5fcd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46984/0 (socket says 172.21.15.112:46984) 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8d4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:44378/0 (socket says 172.21.15.112:44378) 2024-04-25T16:45:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e5fcd640 1 -- 172.21.15.112:0/160713797 learned_addr learned my addr 172.21.15.112:0/160713797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/160713797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 msgr2=0x5589ad8d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/160713797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 msgr2=0x5589ad8d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/160713797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ad540210 con 0x5589ad5074d0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8d4100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00dcae0 tx=0x7fd7d0071dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfe05b83e5b265e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/160713797 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0010720 con 0x5589ad5074d0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7d8022070 con 0x5589ad5074d0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.384+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/160713797 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7fd7d00aff00 con 0x5589ad5074d0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/160713797 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7d00a7040 con 0x5589ad5074d0 2024-04-25T16:45:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 10 client.25679 fetch_fsmap learned FSMap version 25 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589ad60c620 con 0x5589ad5074d0 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/160713797 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7d8022070 con 0x5589ad5074d0 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/160713797 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd7d0018b20 con 0x5589ad5074d0 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 10 client.25679 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 10 client.25679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 20 client.25679 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4e6d80 0x5589ad8e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 4 client.25679 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589ad83c070 con 0x5589ad4e6d80 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4e6d80 0x5589ad8e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.385+0000 7fd7c8ff9640 10 client.25679 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.386+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4e6d80 0x5589ad8e48a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7612bddeb895c0d2 server_cookie=11e1ada4a717b30b in_seq=0 out_seq=0 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.387+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/160713797 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589ad83c070 con 0x5589ad4e6d80 2024-04-25T16:45:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.387+0000 7fd7c8ff9640 10 client.25679 handle_command_reply: tid=0 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.387+0000 7fd7ee9b2740 1 client.25679 shutdown 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 msgr2=0x5589ad8d4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8d4100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00dcae0 tx=0x7fd7d0071dc0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 shutdown_connections 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5281b0 0x5589ad8d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8d4100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4e6d80 0x5589ad8e48a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/160713797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7c8ff9640 0 client.25679 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 >> 172.21.15.112:0/160713797 conn(0x5589ad51c300 msgr2=0x5589ad8c7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 shutdown_connections 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/160713797 wait complete. 2024-04-25T16:45:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.388+0000 7fd7ee9b2740 20 client.25679 trim_cache size 0 max 16384 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.404+0000 7fd7ee9b2740 1 Processor -- start 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.404+0000 7fd7ee9b2740 1 -- start start 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.404+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 0x5589ad8cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.404+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8e9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.404+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad5281b0 0x5589ad8ea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.404+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01b22d0 con 0x5589ad4ff2f0 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01afba0 con 0x5589ad5074d0 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ad9f0 con 0x5589ad5281b0 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 0x5589ad8cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 0x5589ad8cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46990/0 (socket says 172.21.15.112:46990) 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 -- 172.21.15.112:0/3356354315 learned_addr learned my addr 172.21.15.112:0/3356354315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8e9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad5281b0 0x5589ad8ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 -- 172.21.15.112:0/3356354315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad5281b0 msgr2=0x5589ad8ea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad5281b0 0x5589ad8ea530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 -- 172.21.15.112:0/3356354315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 msgr2=0x5589ad8e9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8e9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 -- 172.21.15.112:0/3356354315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ad8c4e90 con 0x5589ad4ff2f0 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 0x5589ad8cbe50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc031440 tx=0x7fd7dc005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79cb63644a6bc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7caffd640 1 -- 172.21.15.112:0/3356354315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc049070 con 0x5589ad4ff2f0 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7caffd640 1 -- 172.21.15.112:0/3356354315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc0054f0 con 0x5589ad4ff2f0 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.405+0000 7fd7caffd640 1 -- 172.21.15.112:0/3356354315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7dc035040 con 0x5589ad4ff2f0 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/3356354315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 msgr2=0x5589ad8cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 0x5589ad8cbe50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc031440 tx=0x7fd7dc005a20 comp rx=0 tx=0).stop 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/3356354315 shutdown_connections 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad5281b0 0x5589ad8ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5074d0 0x5589ad8e9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/3356354315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad4ff2f0 0x5589ad8cbe50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/3356354315 >> 172.21.15.112:0/3356354315 conn(0x5589ad51c300 msgr2=0x5589ad8cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/3356354315 shutdown_connections 2024-04-25T16:45:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.406+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/3356354315 wait complete. 2024-04-25T16:45:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 -- start start 2024-04-25T16:45:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 Processor -- start 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 0x5589ad8d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5281b0 0x5589ad94aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ae060 con 0x5589ad5074d0 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01af4d0 con 0x5589ad5281b0 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7e01ae730 con 0x5589ad4ff2f0 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7ee9b2740 10 client.0 _ll_register_callbacks cb 0x7fd7d000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e4fcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 0x5589ad8d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e4fcb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 0x5589ad8d74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46998/0 (socket says 172.21.15.112:46998) 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/2689047270 learned_addr learned my addr 172.21.15.112:0/2689047270 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e5fcd640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5281b0 0x5589ad94aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/2689047270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 msgr2=0x5589ad8d6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.407+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5281b0 msgr2=0x5589ad94aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5281b0 0x5589ad94aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7e4fcb640 1 -- 172.21.15.112:0/2689047270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ad8d7d10 con 0x5589ad5074d0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7e4fcb640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 0x5589ad8d74c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00a74c0 tx=0x7fd7d00b0230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0e46bfb656cb02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/2689047270 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7d00a6e80 con 0x5589ad5074d0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/2689047270 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7fd7d00a3100 con 0x5589ad5074d0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7dc0054f0 con 0x5589ad5074d0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/2689047270 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd7d0010720 con 0x5589ad5074d0 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-04-25T16:45:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589ad94c100 con 0x5589ad5074d0 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/2689047270 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7dc0054f0 con 0x5589ad5074d0 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/2689047270 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd7d00a6200 con 0x5589ad5074d0 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 20 client.15987 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4bc1c0 0x5589ad94e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 4 client.15987 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.408+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589ad835ee0 con 0x5589ad4bc1c0 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.409+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4bc1c0 0x5589ad94e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.409+0000 7fd7c8ff9640 10 client.15987 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.409+0000 7fd7e57cc640 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4bc1c0 0x5589ad94e7d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969cb86bbbaef941 server_cookie=d76cf7de0859ada2 in_seq=0 out_seq=0 2024-04-25T16:45:41.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:41 smithi133 ceph-mon[32691]: pgmap v223: 97 pgs: 97 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 379 op/s 2024-04-25T16:45:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:41 smithi116 ceph-mon[32712]: pgmap v223: 97 pgs: 97 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 379 op/s 2024-04-25T16:45:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:41 smithi112 ceph-mon[29122]: pgmap v223: 97 pgs: 97 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 379 op/s 2024-04-25T16:45:41.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.599+0000 7fd7c8ff9640 1 -- 172.21.15.112:0/2689047270 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589ad835ee0 con 0x5589ad4bc1c0 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.599+0000 7fd7c8ff9640 10 client.15987 handle_command_reply: tid=0 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.599+0000 7fd7ee9b2740 1 client.15987 shutdown 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 msgr2=0x5589ad8d74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 0x5589ad8d74c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00a74c0 tx=0x7fd7d00b0230 comp rx=0 tx=0).stop 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 shutdown_connections 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589ad5281b0 0x5589ad94aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589ad5074d0 0x5589ad8d74c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589ad4bc1c0 0x5589ad94e7d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 --2- 172.21.15.112:0/2689047270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589ad4ff2f0 0x5589ad8d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7c8ff9640 0 client.15987 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 >> 172.21.15.112:0/2689047270 conn(0x5589ad51c300 msgr2=0x5589ad8d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 shutdown_connections 2024-04-25T16:45:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 1 -- 172.21.15.112:0/2689047270 wait complete. 2024-04-25T16:45:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.600+0000 7fd7ee9b2740 20 client.15987 trim_cache size 0 max 16384 2024-04-25T16:45:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:45:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:45:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:45:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:45:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.601+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd7cc0afcd0 msgr2=0x7fd7cc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.601+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd7cc0afcd0 0x7fd7cc0b21e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8021000 tx=0x7fd7d801f000 comp rx=0 tx=0).stop 2024-04-25T16:45:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.601+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 msgr2=0x7fd7e019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.601+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 0x7fd7e019ac60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000e9f0 tx=0x7fd7d000eeb0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.601+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 shutdown_connections 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd7cc0afcd0 0x7fd7cc0b21e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7e010e0c0 0x7fd7e019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd7e010b6d0 0x7fd7e019ac60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 --2- 172.21.15.112:0/1756534462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7e0104c70 0x7fd7e019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 >> 172.21.15.112:0/1756534462 conn(0x7fd7e00781b0 msgr2=0x7fd7e0101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 shutdown_connections 2024-04-25T16:45:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:41.602+0000 7fd7e7a30640 1 -- 172.21.15.112:0/1756534462 wait complete. 2024-04-25T16:45:41.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 761709 11.06 MB/sec execute 227 sec latency 112.213 ms 2024-04-25T16:45:42.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 764003 11.07 MB/sec execute 228 sec latency 14.260 ms 2024-04-25T16:45:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:43 smithi133 ceph-mon[32691]: pgmap v224: 97 pgs: 97 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-04-25T16:45:43.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:43 smithi116 ceph-mon[32712]: pgmap v224: 97 pgs: 97 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-04-25T16:45:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:43 smithi112 ceph-mon[29122]: pgmap v224: 97 pgs: 97 active+clean; 81 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 181 op/s 2024-04-25T16:45:43.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 765900 11.06 MB/sec execute 229 sec latency 96.706 ms 2024-04-25T16:45:44.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 767828 11.06 MB/sec execute 230 sec latency 20.174 ms 2024-04-25T16:45:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:45 smithi133 ceph-mon[32691]: pgmap v225: 97 pgs: 97 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 320 op/s 2024-04-25T16:45:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:45 smithi116 ceph-mon[32712]: pgmap v225: 97 pgs: 97 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 320 op/s 2024-04-25T16:45:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:45 smithi112 ceph-mon[29122]: pgmap v225: 97 pgs: 97 active+clean; 70 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 320 op/s 2024-04-25T16:45:45.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 770465 11.09 MB/sec execute 231 sec latency 17.969 ms 2024-04-25T16:45:46.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 772061 11.07 MB/sec execute 232 sec latency 19.264 ms 2024-04-25T16:45:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:47 smithi133 ceph-mon[32691]: pgmap v226: 97 pgs: 97 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-04-25T16:45:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:47 smithi116 ceph-mon[32712]: pgmap v226: 97 pgs: 97 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-04-25T16:45:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:47 smithi112 ceph-mon[29122]: pgmap v226: 97 pgs: 97 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-04-25T16:45:47.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 774472 11.08 MB/sec execute 233 sec latency 14.657 ms 2024-04-25T16:45:48.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 776190 11.06 MB/sec execute 234 sec latency 43.153 ms 2024-04-25T16:45:49.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:49 smithi133 ceph-mon[32691]: pgmap v227: 97 pgs: 97 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2024-04-25T16:45:49.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:49 smithi116 ceph-mon[32712]: pgmap v227: 97 pgs: 97 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2024-04-25T16:45:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:49 smithi112 ceph-mon[29122]: pgmap v227: 97 pgs: 97 active+clean; 73 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 264 op/s 2024-04-25T16:45:49.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 778195 11.07 MB/sec execute 235 sec latency 19.272 ms 2024-04-25T16:45:50.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 780566 11.07 MB/sec execute 236 sec latency 22.388 ms 2024-04-25T16:45:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:51 smithi133 ceph-mon[32691]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 354 op/s 2024-04-25T16:45:51.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:51 smithi116 ceph-mon[32712]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 354 op/s 2024-04-25T16:45:51.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:51 smithi112 ceph-mon[29122]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.4 MiB/s wr, 354 op/s 2024-04-25T16:45:51.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 782390 11.06 MB/sec execute 237 sec latency 14.266 ms 2024-04-25T16:45:52.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 784820 11.08 MB/sec execute 238 sec latency 18.605 ms 2024-04-25T16:45:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:53 smithi133 ceph-mon[32691]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-04-25T16:45:53.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:53 smithi116 ceph-mon[32712]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-04-25T16:45:53.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:53 smithi112 ceph-mon[29122]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-04-25T16:45:53.883 INFO:tasks.workunit.client.0.smithi112.stdout: 1 786243 11.06 MB/sec execute 239 sec latency 52.802 ms 2024-04-25T16:45:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 -- 172.21.15.112:0/2526669057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 msgr2=0x7f1b501109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 --2- 172.21.15.112:0/2526669057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b501109c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009ca0 tx=0x7f1b4802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 -- 172.21.15.112:0/2526669057 shutdown_connections 2024-04-25T16:45:54.706 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 --2- 172.21.15.112:0/2526669057 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b501109c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 --2- 172.21.15.112:0/2526669057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 0x7f1b501024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 --2- 172.21.15.112:0/2526669057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 0x7f1b50101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 -- 172.21.15.112:0/2526669057 >> 172.21.15.112:0/2526669057 conn(0x7f1b500778b0 msgr2=0x7f1b50077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 -- 172.21.15.112:0/2526669057 shutdown_connections 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.707+0000 7f1b54b84640 1 -- 172.21.15.112:0/2526669057 wait complete. 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.708+0000 7f1b54b84640 1 Processor -- start 2024-04-25T16:45:54.707 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.708+0000 7f1b54b84640 1 -- start start 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 0x7f1b5019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 0x7f1b5019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b5019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50113d60 con 0x7f1b5010e5a0 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50113be0 con 0x7f1b501020a0 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50113ee0 con 0x7f1b50109bf0 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b4ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b5019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 0x7f1b5019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 0x7f1b5019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 0x7f1b5019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:60846/0 (socket says 172.21.15.112:60846) 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b4ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b5019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51400/0 (socket says 172.21.15.112:51400) 2024-04-25T16:45:54.709 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b4e59c640 1 -- 172.21.15.112:0/727250819 learned_addr learned my addr 172.21.15.112:0/727250819 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 -- 172.21.15.112:0/727250819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 msgr2=0x7f1b5019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 0x7f1b5019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 -- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 msgr2=0x7f1b5019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b5019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 -- 172.21.15.112:0/727250819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5019f540 con 0x7f1b50109bf0 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b5019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b47fff640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 0x7f1b5019ac40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e9c0 tx=0x7f1b3800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b477fe640 1 -- 172.21.15.112:0/727250819 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800ccc0 con 0x7f1b50109bf0 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.709+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5019f830 con 0x7f1b50109bf0 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.710+0000 7f1b477fe640 1 -- 172.21.15.112:0/727250819 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b380108e0 con 0x7f1b50109bf0 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.710+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b501a71b0 con 0x7f1b50109bf0 2024-04-25T16:45:54.710 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.710+0000 7f1b477fe640 1 -- 172.21.15.112:0/727250819 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38018740 con 0x7f1b50109bf0 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 Processor -- start 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b477fe640 1 -- 172.21.15.112:0/727250819 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f1b38010430 con 0x7f1b50109bf0 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 -- start start 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fef51b0 0x55b570294af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fecc2f0 0x55b570295030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fed44d0 0x55b570295570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.717 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5701fb460 con 0x55b56fed44d0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b5701b5590 con 0x55b56fef51b0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.712+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b5701f7420 con 0x55b56fecc2f0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fed44d0 0x55b570295570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fecc2f0 0x55b570295030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b47fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fed44d0 0x55b570295570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51412/0 (socket says 172.21.15.112:51412) 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b47fff640 1 -- 172.21.15.112:0/1344968839 learned_addr learned my addr 172.21.15.112:0/1344968839 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4ed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fef51b0 0x55b570294af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b477fe640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f1b3c0afc50 0x7f1b3c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f1b3c0afc50 0x7f1b3c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1344968839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fef51b0 msgr2=0x55b570294af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fef51b0 0x55b570294af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1344968839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fed44d0 msgr2=0x55b570295570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fed44d0 0x55b570295570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1344968839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5701c20d0 con 0x55b56fecc2f0 2024-04-25T16:45:54.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b477fe640 1 -- 172.21.15.112:0/727250819 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7f1b380df3a0 con 0x7f1b50109bf0 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fecc2f0 0x55b570295030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009c50 tx=0x7f1b40011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=679fd8673ff2f54f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f1b3c0afc50 0x7f1b3c0b2160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b570291c60 tx=0x7f1b4802a8c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b44ff9640 1 -- 172.21.15.112:0/1344968839 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40004230 con 0x55b56fecc2f0 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b44ff9640 1 -- 172.21.15.112:0/1344968839 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b400043d0 con 0x55b56fecc2f0 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.713+0000 7f1b44ff9640 1 -- 172.21.15.112:0/1344968839 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40019790 con 0x55b56fecc2f0 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1344968839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fecc2f0 msgr2=0x55b570295030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fecc2f0 0x55b570295030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009c50 tx=0x7f1b40011810 comp rx=0 tx=0).stop 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1344968839 shutdown_connections 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fecc2f0 0x55b570295030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fed44d0 0x55b570295570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1344968839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fef51b0 0x55b570294af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1344968839 >> 172.21.15.112:0/1344968839 conn(0x55b56fee9300 msgr2=0x55b5702912a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1344968839 shutdown_connections 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.714+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1344968839 wait complete. 2024-04-25T16:45:54.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 -- start start 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 Processor -- start 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 0x55b57029f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fed44d0 0x55b57029f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 0x55b5702a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5702a1c20 con 0x55b56fef51b0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b570182b60 con 0x55b56fecc2f0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b5702a1da0 con 0x55b56fed44d0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b5ba38740 10 client.0 _ll_register_callbacks cb 0x55b5701a72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b4ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 0x55b5702a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 0x55b57029f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 0x55b57029f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:60860/0 (socket says 172.21.15.112:60860) 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b4ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 0x55b5702a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51426/0 (socket says 172.21.15.112:51426) 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b4e59c640 1 -- 172.21.15.112:0/288679509 learned_addr learned my addr 172.21.15.112:0/288679509 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b47fff640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fed44d0 0x55b57029f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.715+0000 7f1b4e59c640 1 -- 172.21.15.112:0/288679509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fed44d0 msgr2=0x55b57029f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fed44d0 0x55b57029f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b4e59c640 1 -- 172.21.15.112:0/288679509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 msgr2=0x55b5702a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 0x55b5702a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b4e59c640 1 -- 172.21.15.112:0/288679509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56ff0d210 con 0x55b56fecc2f0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 0x55b5702a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 0x55b57029f120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009820 tx=0x7f1b40004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6546d5f085f829b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b35ffb640 1 -- 172.21.15.112:0/288679509 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40029020 con 0x55b56fecc2f0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b400043d0 con 0x55b56fecc2f0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b35ffb640 1 -- 172.21.15.112:0/288679509 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7f1b400037f0 con 0x55b56fecc2f0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b35ffb640 1 -- 172.21.15.112:0/288679509 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b40034050 con 0x55b56fecc2f0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b5ba38740 10 client.25697 fetch_fsmap learned FSMap version 25 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b56ffd9620 con 0x55b56fecc2f0 2024-04-25T16:45:54.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b35ffb640 1 -- 172.21.15.112:0/288679509 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b400043d0 con 0x55b56fecc2f0 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.716+0000 7f1b35ffb640 1 -- 172.21.15.112:0/288679509 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1b40004b70 con 0x55b56fecc2f0 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b5ba38740 10 client.25697 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b5ba38740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b5ba38740 20 client.25697 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56feb3d80 0x55b5702b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b5ba38740 4 client.25697 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b570209070 con 0x55b56feb3d80 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b47fff640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56feb3d80 0x55b5702b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b35ffb640 10 client.25697 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.717+0000 7f1b47fff640 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56feb3d80 0x55b5702b1630 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2247fecd403334b8 server_cookie=7f67e07efbed022 in_seq=0 out_seq=0 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.718+0000 7f1b35ffb640 1 -- 172.21.15.112:0/288679509 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b570209070 con 0x55b56feb3d80 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.718+0000 7f1b35ffb640 10 client.25697 handle_command_reply: tid=0 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 client.25697 shutdown 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 msgr2=0x55b57029f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.722 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 0x55b57029f120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009820 tx=0x7f1b40004360 comp rx=0 tx=0).stop 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 shutdown_connections 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56feb3d80 0x55b5702b1630 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fed44d0 0x55b57029f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fef51b0 0x55b5702a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/288679509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fecc2f0 0x55b57029f120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b35ffb640 0 client.25697 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 >> 172.21.15.112:0/288679509 conn(0x55b56fee9300 msgr2=0x55b570294920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 shutdown_connections 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 1 -- 172.21.15.112:0/288679509 wait complete. 2024-04-25T16:45:54.723 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.719+0000 7f1b5ba38740 20 client.25697 trim_cache size 0 max 16384 2024-04-25T16:45:54.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 Processor -- start 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 -- start start 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b570294f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b5702b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501b2380 con 0x55b56fecc2f0 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501afc50 con 0x55b56fed44d0 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501adaa0 con 0x55b56fef51b0 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51428/0 (socket says 172.21.15.112:51428) 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1403259606 learned_addr learned my addr 172.21.15.112:0/1403259606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b47fff640 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b570294f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b5702b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1403259606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 msgr2=0x55b5702b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b5702b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1403259606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 msgr2=0x55b570294f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.735+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b570294f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.736+0000 7f1b4e59c640 1 -- 172.21.15.112:0/1403259606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b57029fe40 con 0x55b56fecc2f0 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.736+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b5702b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.736+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a5010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b40019460 tx=0x7f1b40004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b2fdc7539c321e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.736+0000 7f1b37fff640 1 -- 172.21.15.112:0/1403259606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40029040 con 0x55b56fecc2f0 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.736+0000 7f1b37fff640 1 -- 172.21.15.112:0/1403259606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b40009850 con 0x55b56fecc2f0 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.736+0000 7f1b37fff640 1 -- 172.21.15.112:0/1403259606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40003850 con 0x55b56fecc2f0 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1403259606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 msgr2=0x55b5702a5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a5010 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b40019460 tx=0x7f1b40004fd0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1403259606 shutdown_connections 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b5702b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b570294f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/1403259606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a5010 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1403259606 >> 172.21.15.112:0/1403259606 conn(0x55b56fee9300 msgr2=0x55b570294880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1403259606 shutdown_connections 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- 172.21.15.112:0/1403259606 wait complete. 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- start start 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 Processor -- start 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b5702a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b570317870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501ae110 con 0x55b56fecc2f0 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501af580 con 0x55b56fed44d0 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501ae7e0 con 0x55b56fef51b0 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b5ba38740 10 client.0 _ll_register_callbacks cb 0x7f1b40009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b4e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51438/0 (socket says 172.21.15.112:51438) 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b4e59c640 1 -- 172.21.15.112:0/2536765415 learned_addr learned my addr 172.21.15.112:0/2536765415 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.737+0000 7f1b47fff640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b5702a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b570317870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4e59c640 1 -- 172.21.15.112:0/2536765415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 msgr2=0x55b570317870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b570317870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4e59c640 1 -- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 msgr2=0x55b5702a88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b5702a88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4e59c640 1 -- 172.21.15.112:0/2536765415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b570317db0 con 0x55b56fecc2f0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4ed9d640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b570317870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b4e59c640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a8370 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009820 tx=0x7f1b400056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3db127fb80826f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b35ffb640 1 -- 172.21.15.112:0/2536765415 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40044020 con 0x55b56fecc2f0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b35ffb640 1 -- 172.21.15.112:0/2536765415 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8556+0+0 (secure 0 0 0) 0x7f1b40003b80 con 0x55b56fecc2f0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b570202ee0 con 0x55b56fecc2f0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b35ffb640 1 -- 172.21.15.112:0/2536765415 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1b40029040 con 0x55b56fecc2f0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b5ba38740 10 client.16005 fetch_fsmap learned FSMap version 25 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b570318f50 con 0x55b56fecc2f0 2024-04-25T16:45:54.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.738+0000 7f1b35ffb640 1 -- 172.21.15.112:0/2536765415 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b570202ee0 con 0x55b56fecc2f0 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b35ffb640 1 -- 172.21.15.112:0/2536765415 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1b400257a0 con 0x55b56fecc2f0 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b5ba38740 10 client.16005 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b5ba38740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b5ba38740 20 client.16005 populate_metadata read hostname 'smithi112' 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56fe891c0 0x55b57031b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b5ba38740 4 client.16005 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b57017e5f0 con 0x55b56fe891c0 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b47fff640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56fe891c0 0x55b57031b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b47fff640 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56fe891c0 0x55b57031b5c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ae337154adfb303 server_cookie=e5229e84f3239e9e in_seq=0 out_seq=0 2024-04-25T16:45:54.743 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.739+0000 7f1b35ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:54.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 788393 11.07 MB/sec execute 240 sec latency 49.789 ms 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b35ffb640 1 -- 172.21.15.112:0/2536765415 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b57017e5f0 con 0x55b56fe891c0 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b35ffb640 10 client.16005 handle_command_reply: tid=0 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 client.16005 shutdown 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 msgr2=0x55b5702a8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a8370 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009820 tx=0x7f1b400056f0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 shutdown_connections 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b56fe891c0 0x55b57031b5c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b35ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b56fef51b0 0x55b570317870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b56fed44d0 0x55b5702a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 --2- 172.21.15.112:0/2536765415 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56fecc2f0 0x55b5702a8370 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 >> 172.21.15.112:0/2536765415 conn(0x55b56fee9300 msgr2=0x55b5702a05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 shutdown_connections 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 1 -- 172.21.15.112:0/2536765415 wait complete. 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.899+0000 7f1b5ba38740 20 client.16005 trim_cache size 0 max 16384 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:45:54.899 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:45:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f1b3c0afc50 msgr2=0x7f1b3c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f1b3c0afc50 0x7f1b3c0b2160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55b570291c60 tx=0x7f1b4802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 msgr2=0x7f1b5019ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:45:54.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 0x7f1b5019ac40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800e9c0 tx=0x7f1b3800ee80 comp rx=0 tx=0).stop 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 shutdown_connections 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f1b3c0afc50 0x7f1b3c0b2160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b50109bf0 0x7f1b5019ac40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b5010e5a0 0x7f1b5019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 --2- 172.21.15.112:0/727250819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b501020a0 0x7f1b5019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 >> 172.21.15.112:0/727250819 conn(0x7f1b500778b0 msgr2=0x7f1b5010a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 shutdown_connections 2024-04-25T16:45:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:45:54.900+0000 7f1b54b84640 1 -- 172.21.15.112:0/727250819 wait complete. 2024-04-25T16:45:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:55 smithi133 ceph-mon[32691]: pgmap v230: 97 pgs: 97 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2024-04-25T16:45:55.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:55 smithi116 ceph-mon[32712]: pgmap v230: 97 pgs: 97 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2024-04-25T16:45:55.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:55 smithi112 ceph-mon[29122]: pgmap v230: 97 pgs: 97 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 324 op/s 2024-04-25T16:45:55.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 790055 11.05 MB/sec execute 241 sec latency 58.006 ms 2024-04-25T16:45:56.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 792368 11.06 MB/sec execute 242 sec latency 16.779 ms 2024-04-25T16:45:57.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:57 smithi133 ceph-mon[32691]: pgmap v231: 97 pgs: 97 active+clean; 91 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 222 op/s 2024-04-25T16:45:57.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:57 smithi116 ceph-mon[32712]: pgmap v231: 97 pgs: 97 active+clean; 91 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 222 op/s 2024-04-25T16:45:57.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:57 smithi112 ceph-mon[29122]: pgmap v231: 97 pgs: 97 active+clean; 91 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.3 MiB/s wr, 222 op/s 2024-04-25T16:45:57.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 794694 11.07 MB/sec execute 243 sec latency 29.911 ms 2024-04-25T16:45:58.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 796244 11.05 MB/sec execute 244 sec latency 52.150 ms 2024-04-25T16:45:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:45:59 smithi133 ceph-mon[32691]: pgmap v232: 97 pgs: 97 active+clean; 91 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-04-25T16:45:59.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:45:59 smithi116 ceph-mon[32712]: pgmap v232: 97 pgs: 97 active+clean; 91 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-04-25T16:45:59.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:45:59 smithi112 ceph-mon[29122]: pgmap v232: 97 pgs: 97 active+clean; 91 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 185 op/s 2024-04-25T16:45:59.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 798678 11.06 MB/sec execute 245 sec latency 24.866 ms 2024-04-25T16:46:00.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 799880 11.04 MB/sec execute 246 sec latency 25.353 ms 2024-04-25T16:46:01.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:01 smithi133 ceph-mon[32691]: pgmap v233: 97 pgs: 97 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 364 op/s 2024-04-25T16:46:01.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:01 smithi116 ceph-mon[32712]: pgmap v233: 97 pgs: 97 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 364 op/s 2024-04-25T16:46:01.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:01 smithi112 ceph-mon[29122]: pgmap v233: 97 pgs: 97 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.8 MiB/s wr, 364 op/s 2024-04-25T16:46:01.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 801764 11.04 MB/sec execute 247 sec latency 24.435 ms 2024-04-25T16:46:02.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 803695 11.04 MB/sec execute 248 sec latency 17.464 ms 2024-04-25T16:46:03.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:03 smithi133 ceph-mon[32691]: pgmap v234: 97 pgs: 97 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-04-25T16:46:03.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:03 smithi116 ceph-mon[32712]: pgmap v234: 97 pgs: 97 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-04-25T16:46:03.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:03 smithi112 ceph-mon[29122]: pgmap v234: 97 pgs: 97 active+clean; 74 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 274 op/s 2024-04-25T16:46:03.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 806025 11.05 MB/sec execute 249 sec latency 36.981 ms 2024-04-25T16:46:04.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 807442 11.04 MB/sec execute 250 sec latency 65.508 ms 2024-04-25T16:46:05.396 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:05 smithi112 ceph-mon[29122]: pgmap v235: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 386 op/s 2024-04-25T16:46:05.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:05 smithi133 ceph-mon[32691]: pgmap v235: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 386 op/s 2024-04-25T16:46:05.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:05 smithi116 ceph-mon[32712]: pgmap v235: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.9 MiB/s wr, 386 op/s 2024-04-25T16:46:05.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 809742 11.05 MB/sec execute 251 sec latency 18.313 ms 2024-04-25T16:46:06.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 810385 11.01 MB/sec execute 252 sec latency 50.582 ms 2024-04-25T16:46:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:07 smithi133 ceph-mon[32691]: pgmap v236: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-04-25T16:46:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:07 smithi133 ceph-mon[32691]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T16:46:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:07 smithi116 ceph-mon[32712]: pgmap v236: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-04-25T16:46:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:07 smithi116 ceph-mon[32712]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T16:46:07.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:07 smithi112 ceph-mon[29122]: pgmap v236: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 333 op/s 2024-04-25T16:46:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:07 smithi112 ceph-mon[29122]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T16:46:07.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 811111 10.98 MB/sec execute 253 sec latency 39.338 ms 2024-04-25T16:46:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.052+0000 7f391cae0640 1 -- 172.21.15.112:0/1404256190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3918108f00 msgr2=0x7f3918102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.052+0000 7f391cae0640 1 --2- 172.21.15.112:0/1404256190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3918108f00 0x7f3918102b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f390c00b9f0 tx=0x7f390c02c0a0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 -- 172.21.15.112:0/1404256190 shutdown_connections 2024-04-25T16:46:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 --2- 172.21.15.112:0/1404256190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39181031c0 0x7f391810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.051 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 --2- 172.21.15.112:0/1404256190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3918108f00 0x7f3918102b50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 --2- 172.21.15.112:0/1404256190 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 -- 172.21.15.112:0/1404256190 >> 172.21.15.112:0/1404256190 conn(0x7f39180781b0 msgr2=0x7f39180785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 -- 172.21.15.112:0/1404256190 shutdown_connections 2024-04-25T16:46:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.053+0000 7f391cae0640 1 -- 172.21.15.112:0/1404256190 wait complete. 2024-04-25T16:46:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.054+0000 7f391cae0640 1 Processor -- start 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.054+0000 7f391cae0640 1 -- start start 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391cae0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39181031c0 0x7f39181985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391cae0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391cae0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3918108f00 0x7f391819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391cae0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3918111b60 con 0x7f3918108a60 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391cae0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f39181119e0 con 0x7f39181031c0 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391cae0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3918111ce0 con 0x7f3918108f00 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58938/0 (socket says 172.21.15.112:58938) 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391659c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39181031c0 0x7f39181985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 -- 172.21.15.112:0/2434633296 learned_addr learned my addr 172.21.15.112:0/2434633296 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3916d9d640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3918108f00 0x7f391819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 -- 172.21.15.112:0/2434633296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3918108f00 msgr2=0x7f391819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3918108f00 0x7f391819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 -- 172.21.15.112:0/2434633296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39181031c0 msgr2=0x7f39181985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39181031c0 0x7f39181985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f3915d9b640 1 -- 172.21.15.112:0/2434633296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391819d400 con 0x7f3918108a60 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.055+0000 7f391659c640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39181031c0 0x7f39181985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.056+0000 7f3915d9b640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918198b00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.056+0000 7f39077fe640 1 -- 172.21.15.112:0/2434633296 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c041030 con 0x7f3918108a60 2024-04-25T16:46:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.056+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391819d690 con 0x7f3918108a60 2024-04-25T16:46:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.056+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391819dbd0 con 0x7f3918108a60 2024-04-25T16:46:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.056+0000 7f39077fe640 1 -- 172.21.15.112:0/2434633296 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c02ccf0 con 0x7f3918108a60 2024-04-25T16:46:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.056+0000 7f39077fe640 1 -- 172.21.15.112:0/2434633296 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c041030 con 0x7f3918108a60 2024-04-25T16:46:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.059+0000 7f39077fe640 1 -- 172.21.15.112:0/2434633296 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f390c044070 con 0x7f3918108a60 2024-04-25T16:46:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.060+0000 7f39077fe640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f39000b0050 0x7f39000b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.060+0000 7f39077fe640 1 -- 172.21.15.112:0/2434633296 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8644+0+0 (secure 0 0 0) 0x7f390c042070 con 0x7f3918108a60 2024-04-25T16:46:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.060+0000 7f391659c640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f39000b0050 0x7f39000b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 Processor -- start 2024-04-25T16:46:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 -- start start 2024-04-25T16:46:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f391659c640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f39000b0050 0x7f39000b2560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f391806c590 tx=0x7f390800b390 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd669c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd66f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd67440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc6dccd460 con 0x55cc6d9a64d0 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cc6dc87590 con 0x55cc6d99e2f0 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f392398b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cc6dcc9420 con 0x55cc6d9c71b0 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f3916d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd67440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f391659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd669c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f3916d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd67440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58946/0 (socket says 172.21.15.112:58946) 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f3916d9d640 1 -- 172.21.15.112:0/79471920 learned_addr learned my addr 172.21.15.112:0/79471920 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.061+0000 7f3915d9b640 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd66f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f391659c640 1 -- 172.21.15.112:0/79471920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d99e2f0 msgr2=0x55cc6dd66f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f391659c640 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd66f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f391659c640 1 -- 172.21.15.112:0/79471920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 msgr2=0x55cc6dd67440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f391659c640 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd67440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f391659c640 1 -- 172.21.15.112:0/79471920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6dc940d0 con 0x55cc6d9c71b0 2024-04-25T16:46:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f391659c640 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd669c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3918103750 tx=0x7f3908013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e359d6cab26c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f3904ff9640 1 -- 172.21.15.112:0/79471920 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390801c780 con 0x55cc6d9c71b0 2024-04-25T16:46:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f3904ff9640 1 -- 172.21.15.112:0/79471920 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390801cdd0 con 0x55cc6d9c71b0 2024-04-25T16:46:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.062+0000 7f3904ff9640 1 -- 172.21.15.112:0/79471920 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390801d3b0 con 0x55cc6d9c71b0 2024-04-25T16:46:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 -- 172.21.15.112:0/79471920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 msgr2=0x55cc6dd669c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd669c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3918103750 tx=0x7f3908013000 comp rx=0 tx=0).stop 2024-04-25T16:46:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 -- 172.21.15.112:0/79471920 shutdown_connections 2024-04-25T16:46:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd67440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd66f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 --2- 172.21.15.112:0/79471920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd669c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.064+0000 7f392398b740 1 -- 172.21.15.112:0/79471920 >> 172.21.15.112:0/79471920 conn(0x55cc6d9bb300 msgr2=0x55cc6dd632a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.065+0000 7f392398b740 1 -- 172.21.15.112:0/79471920 shutdown_connections 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.065+0000 7f392398b740 1 -- 172.21.15.112:0/79471920 wait complete. 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 -- start start 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 Processor -- start 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd75a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55cc6dd780e0 con 0x55cc6d9a64d0 2024-04-25T16:46:08.065 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cc6dc54b60 con 0x55cc6d9c71b0 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55cc6dd78260 con 0x55cc6d99e2f0 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f392398b740 10 client.0 _ll_register_callbacks cb 0x55cc6dc792b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f3915d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd75a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f391659c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f3915d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd75a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58954/0 (socket says 172.21.15.112:58954) 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f3915d9b640 1 -- 172.21.15.112:0/3731196334 learned_addr learned my addr 172.21.15.112:0/3731196334 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.066+0000 7f3916d9d640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f3915d9b640 1 -- 172.21.15.112:0/3731196334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d99e2f0 msgr2=0x55cc6dd754d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f3915d9b640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd754d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f3915d9b640 1 -- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9c71b0 msgr2=0x55cc6dd6c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f3915d9b640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f3915d9b640 1 -- 172.21.15.112:0/3731196334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6d9df210 con 0x55cc6d9a64d0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f3915d9b640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd75a10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f390c0c9380 tx=0x7f390c0c8d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5fb1e03fb0b3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f38fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f38fdffb640 1 -- 172.21.15.112:0/3731196334 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c041040 con 0x55cc6d9a64d0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390801cdd0 con 0x55cc6d9a64d0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.067+0000 7f38fdffb640 1 -- 172.21.15.112:0/3731196334 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8644+0+0 (secure 0 0 0) 0x7f390c0d1dc0 con 0x55cc6d9a64d0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f391659c640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f38fdffb640 1 -- 172.21.15.112:0/3731196334 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f390c0c9af0 con 0x55cc6d9a64d0 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 10 client.16017 fetch_fsmap learned FSMap version 25 2024-04-25T16:46:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc6daab620 con 0x55cc6d9a64d0 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f38fdffb640 1 -- 172.21.15.112:0/3731196334 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390c0c93f0 con 0x55cc6d9a64d0 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f38fdffb640 1 -- 172.21.15.112:0/3731196334 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f390c0e3ce0 con 0x55cc6d9a64d0 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 10 client.16017 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 20 client.16017 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d985d80 0x55cc6dd8c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 4 client.16017 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.068+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f390801cdd0 con 0x55cc6d985d80 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.069+0000 7f391659c640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d985d80 0x55cc6dd8c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.069+0000 7f391659c640 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d985d80 0x55cc6dd8c020 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61862bfb594bd3d5 server_cookie=af01b533f63abdbc in_seq=0 out_seq=0 2024-04-25T16:46:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.069+0000 7f38fdffb640 10 client.16017 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.071+0000 7f38fdffb640 1 -- 172.21.15.112:0/3731196334 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f390801cdd0 con 0x55cc6d985d80 2024-04-25T16:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.071+0000 7f38fdffb640 10 client.16017 handle_command_reply: tid=0 2024-04-25T16:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.071+0000 7f392398b740 1 client.16017 shutdown 2024-04-25T16:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 msgr2=0x55cc6dd75a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd75a10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f390c0c9380 tx=0x7f390c0c8d40 comp rx=0 tx=0).stop 2024-04-25T16:46:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 shutdown_connections 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd75a10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d985d80 0x55cc6dd8c020 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 --2- 172.21.15.112:0/3731196334 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd754d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 >> 172.21.15.112:0/3731196334 conn(0x55cc6d9bb300 msgr2=0x55cc6dd69160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f38fdffb640 0 client.16017 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 shutdown_connections 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 1 -- 172.21.15.112:0/3731196334 wait complete. 2024-04-25T16:46:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.072+0000 7f392398b740 20 client.16017 trim_cache size 0 max 16384 2024-04-25T16:46:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.100+0000 7f392398b740 1 Processor -- start 2024-04-25T16:46:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.100+0000 7f392398b740 1 -- start start 2024-04-25T16:46:08.099 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f392398b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f392398b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd6ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f392398b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f392398b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38f8011140 con 0x55cc6d99e2f0 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f392398b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38f800ea10 con 0x55cc6d9a64d0 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f392398b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38f800c880 con 0x55cc6d9c71b0 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f391659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.100 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f391659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd742a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58960/0 (socket says 172.21.15.112:58960) 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f391659c640 1 -- 172.21.15.112:0/249290398 learned_addr learned my addr 172.21.15.112:0/249290398 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f3916d9d640 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.101+0000 7f3915d9b640 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd6ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f391659c640 1 -- 172.21.15.112:0/249290398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 msgr2=0x55cc6dd6f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f391659c640 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f391659c640 1 -- 172.21.15.112:0/249290398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 msgr2=0x55cc6dd6ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f391659c640 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd6ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f391659c640 1 -- 172.21.15.112:0/249290398 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc6dd8c550 con 0x55cc6d99e2f0 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f391659c640 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd742a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3908011010 tx=0x7f3908016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8993cfa1e7fc437e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:08.101 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f38fffff640 1 -- 172.21.15.112:0/249290398 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390801eaa0 con 0x55cc6d99e2f0 2024-04-25T16:46:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.102+0000 7f38fffff640 1 -- 172.21.15.112:0/249290398 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f390801ec40 con 0x55cc6d99e2f0 2024-04-25T16:46:08.102 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.103+0000 7f38fffff640 1 -- 172.21.15.112:0/249290398 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39080055d0 con 0x55cc6d99e2f0 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.103+0000 7f392398b740 1 -- 172.21.15.112:0/249290398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 msgr2=0x55cc6dd742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.103+0000 7f392398b740 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd742a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3908011010 tx=0x7f3908016000 comp rx=0 tx=0).stop 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 -- 172.21.15.112:0/249290398 shutdown_connections 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd6ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 --2- 172.21.15.112:0/249290398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd742a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 -- 172.21.15.112:0/249290398 >> 172.21.15.112:0/249290398 conn(0x55cc6d9bb300 msgr2=0x55cc6dd73b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 -- 172.21.15.112:0/249290398 shutdown_connections 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 -- 172.21.15.112:0/249290398 wait complete. 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.104+0000 7f392398b740 1 -- start start 2024-04-25T16:46:08.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 Processor -- start 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd86820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd81b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38f800ced0 con 0x55cc6d99e2f0 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f3915d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd81b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f391659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd86820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f3915d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd81b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:55982/0 (socket says 172.21.15.112:55982) 2024-04-25T16:46:08.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f3915d9b640 1 -- 172.21.15.112:0/3603582532 learned_addr learned my addr 172.21.15.112:0/3603582532 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f391659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd86820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58972/0 (socket says 172.21.15.112:58972) 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38f800e340 con 0x55cc6d9a64d0 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.105+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38f800d5a0 con 0x55cc6d9c71b0 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f392398b740 10 client.0 _ll_register_callbacks cb 0x7f390c0c52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 -- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 msgr2=0x55cc6dd81b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd81b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 -- 172.21.15.112:0/3603582532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 msgr2=0x55cc6dd86820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd86820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 -- 172.21.15.112:0/3603582532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc6dd82910 con 0x55cc6d9c71b0 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f391659c640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd86820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:08.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f3916d9d640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd821a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f39100098d0 tx=0x7f391000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b1906dda3c0fa37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.106+0000 7f38fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:46:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.107+0000 7f38fdffb640 1 -- 172.21.15.112:0/3603582532 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910020020 con 0x55cc6d9c71b0 2024-04-25T16:46:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.107+0000 7f38fdffb640 1 -- 172.21.15.112:0/3603582532 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8644+0+0 (secure 0 0 0) 0x7f391001c870 con 0x55cc6d9c71b0 2024-04-25T16:46:08.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.107+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390801ec40 con 0x55cc6d9c71b0 2024-04-25T16:46:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.107+0000 7f38fdffb640 1 -- 172.21.15.112:0/3603582532 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f391000ce30 con 0x55cc6d9c71b0 2024-04-25T16:46:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.108+0000 7f392398b740 10 client.25651 fetch_fsmap learned FSMap version 25 2024-04-25T16:46:08.108 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.108+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cc6ddf4390 con 0x55cc6d9c71b0 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.108+0000 7f38fdffb640 1 -- 172.21.15.112:0/3603582532 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39100253f0 con 0x55cc6d9c71b0 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.108+0000 7f38fdffb640 1 -- 172.21.15.112:0/3603582532 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f391002d020 con 0x55cc6d9c71b0 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.109+0000 7f392398b740 10 client.25651 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.109+0000 7f392398b740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.109+0000 7f392398b740 20 client.25651 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.109+0000 7f392398b740 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d95b1c0 0x55cc6ddf6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.109+0000 7f392398b740 4 client.25651 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.109+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f390801ec40 con 0x55cc6d95b1c0 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.110+0000 7f391659c640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d95b1c0 0x55cc6ddf6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.111+0000 7f391659c640 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d95b1c0 0x55cc6ddf6bb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2865d466c92085dd server_cookie=c45df03dd51a4430 in_seq=0 out_seq=0 2024-04-25T16:46:08.109 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.111+0000 7f38fdffb640 10 client.25651 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.316+0000 7f38fdffb640 1 -- 172.21.15.112:0/3603582532 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f390801ec40 con 0x55cc6d95b1c0 2024-04-25T16:46:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.316+0000 7f38fdffb640 10 client.25651 handle_command_reply: tid=0 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.316+0000 7f392398b740 1 client.25651 shutdown 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.316+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 msgr2=0x55cc6dd821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.316+0000 7f392398b740 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd821a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f39100098d0 tx=0x7f391000c7a0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 shutdown_connections 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55cc6d95b1c0 0x55cc6ddf6bb0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f38fdffb640 0 client.25651 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cc6d9c71b0 0x55cc6dd821a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cc6d9a64d0 0x55cc6dd81b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 --2- 172.21.15.112:0/3603582532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55cc6d99e2f0 0x55cc6dd86820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 >> 172.21.15.112:0/3603582532 conn(0x55cc6d9bb300 msgr2=0x55cc6dd672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 shutdown_connections 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 1 -- 172.21.15.112:0/3603582532 wait complete. 2024-04-25T16:46:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.317+0000 7f392398b740 20 client.25651 trim_cache size 0 max 16384 2024-04-25T16:46:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:46:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:46:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:46:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f39000b0050 msgr2=0x7f39000b2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f39000b0050 0x7f39000b2560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f391806c590 tx=0x7f390800b390 comp rx=0 tx=0).stop 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 msgr2=0x7f3918198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918198b00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f390c000c00 tx=0x7f390c009680 comp rx=0 tx=0).stop 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 shutdown_connections 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f39000b0050 0x7f39000b2560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3918108f00 0x7f391819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3918108a60 0x7f3918198b00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 --2- 172.21.15.112:0/2434633296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f39181031c0 0x7f39181985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 >> 172.21.15.112:0/2434633296 conn(0x7f39180781b0 msgr2=0x7f3918104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 shutdown_connections 2024-04-25T16:46:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:08.318+0000 7f391cae0640 1 -- 172.21.15.112:0/2434633296 wait complete. 2024-04-25T16:46:08.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 812864 10.97 MB/sec execute 254 sec latency 109.943 ms 2024-04-25T16:46:09.008 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setuser ceph since I am not root 2024-04-25T16:46:09.008 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setgroup ceph since I am not root 2024-04-25T16:46:09.008 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:08 smithi116 ceph-mds[62762]: -- 172.21.15.116:0/4235770867 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55993708fa40 con 0x5599370d7600 2024-04-25T16:46:09.008 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: starting mds.e at 2024-04-25T16:46:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:09 smithi133 ceph-mon[32691]: pgmap v238: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-04-25T16:46:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:09 smithi133 ceph-mon[32691]: daemon mds.e restarted 2024-04-25T16:46:09.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:09 smithi116 ceph-mon[32712]: pgmap v238: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-04-25T16:46:09.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:09 smithi116 ceph-mon[32712]: daemon mds.e restarted 2024-04-25T16:46:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:09 smithi112 ceph-mon[29122]: pgmap v238: 97 pgs: 97 active+clean; 82 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.0 MiB/s wr, 354 op/s 2024-04-25T16:46:09.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:09 smithi112 ceph-mon[29122]: daemon mds.e restarted 2024-04-25T16:46:09.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 814047 10.95 MB/sec execute 255 sec latency 47.610 ms 2024-04-25T16:46:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:10 smithi133 ceph-mon[32691]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T16:46:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:10 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6834/2941602497,v1:172.21.15.116:6835/2941602497] up:boot 2024-04-25T16:46:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:10 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:46:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:10 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:46:10.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:10 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:46:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:10 smithi116 ceph-mon[32712]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T16:46:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:10 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6834/2941602497,v1:172.21.15.116:6835/2941602497] up:boot 2024-04-25T16:46:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:10 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:46:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:10 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:46:10.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:10 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:46:10.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:10 smithi112 ceph-mon[29122]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T16:46:10.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:10 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6834/2941602497,v1:172.21.15.116:6835/2941602497] up:boot 2024-04-25T16:46:10.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:10 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:46:10.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:10 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:46:10.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:10 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:46:10.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 816608 10.97 MB/sec execute 256 sec latency 20.536 ms 2024-04-25T16:46:11.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:11 smithi133 ceph-mon[32691]: pgmap v240: 97 pgs: 97 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 408 op/s 2024-04-25T16:46:11.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:11 smithi116 ceph-mon[32712]: pgmap v240: 97 pgs: 97 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 408 op/s 2024-04-25T16:46:11.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:11 smithi112 ceph-mon[29122]: pgmap v240: 97 pgs: 97 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 408 op/s 2024-04-25T16:46:11.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 818155 10.96 MB/sec execute 257 sec latency 15.443 ms 2024-04-25T16:46:12.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 820454 10.97 MB/sec execute 258 sec latency 14.827 ms 2024-04-25T16:46:13.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:13 smithi112 ceph-mon[29122]: pgmap v241: 97 pgs: 97 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-04-25T16:46:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:13 smithi112 ceph-mon[29122]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T16:46:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:13 smithi133 ceph-mon[32691]: pgmap v241: 97 pgs: 97 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-04-25T16:46:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:13 smithi133 ceph-mon[32691]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T16:46:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:13 smithi116 ceph-mon[32712]: pgmap v241: 97 pgs: 97 active+clean; 63 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-04-25T16:46:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:13 smithi116 ceph-mon[32712]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T16:46:13.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 821981 10.95 MB/sec execute 259 sec latency 18.863 ms 2024-04-25T16:46:14.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 823992 10.96 MB/sec execute 260 sec latency 93.604 ms 2024-04-25T16:46:15.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:15 smithi112 ceph-mon[29122]: pgmap v243: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 394 op/s 2024-04-25T16:46:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:15 smithi133 ceph-mon[32691]: pgmap v243: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 394 op/s 2024-04-25T16:46:15.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:15 smithi116 ceph-mon[32712]: pgmap v243: 97 pgs: 97 active+clean; 70 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 394 op/s 2024-04-25T16:46:15.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 825663 10.94 MB/sec execute 261 sec latency 53.692 ms 2024-04-25T16:46:16.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 827888 10.95 MB/sec execute 262 sec latency 13.894 ms 2024-04-25T16:46:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:17 smithi112 ceph-mon[29122]: pgmap v244: 97 pgs: 97 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 377 op/s 2024-04-25T16:46:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:17 smithi133 ceph-mon[32691]: pgmap v244: 97 pgs: 97 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 377 op/s 2024-04-25T16:46:17.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:17 smithi116 ceph-mon[32712]: pgmap v244: 97 pgs: 97 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.3 MiB/s wr, 377 op/s 2024-04-25T16:46:17.884 INFO:tasks.workunit.client.0.smithi112.stdout: 1 830121 10.95 MB/sec execute 263 sec latency 15.533 ms 2024-04-25T16:46:18.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 831754 10.94 MB/sec execute 264 sec latency 20.340 ms 2024-04-25T16:46:19.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:19 smithi112 ceph-mon[29122]: pgmap v245: 97 pgs: 97 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-04-25T16:46:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:19 smithi133 ceph-mon[32691]: pgmap v245: 97 pgs: 97 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-04-25T16:46:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:19 smithi116 ceph-mon[32712]: pgmap v245: 97 pgs: 97 active+clean; 70 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 136 op/s 2024-04-25T16:46:19.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 833542 10.94 MB/sec execute 265 sec latency 57.040 ms 2024-04-25T16:46:20.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 835482 10.93 MB/sec execute 266 sec latency 16.024 ms 2024-04-25T16:46:21.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:21 smithi112 ceph-mon[29122]: pgmap v246: 97 pgs: 97 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 -- 172.21.15.112:0/582999751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 msgr2=0x7fd9a410f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/582999751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a410f4f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009ca0 tx=0x7fd99c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 -- 172.21.15.112:0/582999751 shutdown_connections 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/582999751 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a410f4f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/582999751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108f00 0x7fd9a4102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/582999751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a4108a60 0x7fd9a4102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 -- 172.21.15.112:0/582999751 >> 172.21.15.112:0/582999751 conn(0x7fd9a40781b0 msgr2=0x7fd9a40785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 -- 172.21.15.112:0/582999751 shutdown_connections 2024-04-25T16:46:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 -- 172.21.15.112:0/582999751 wait complete. 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.421+0000 7fd9a98f9640 1 Processor -- start 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 -- start start 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108a60 0x7fd9a4199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a4108f00 0x7fd9a419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4112ca0 con 0x7fd9a41031c0 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4112b20 con 0x7fd9a4108a60 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a98f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4112e20 con 0x7fd9a4108f00 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a4199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33460/0 (socket says 172.21.15.112:33460) 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 -- 172.21.15.112:0/3064809590 learned_addr learned my addr 172.21.15.112:0/3064809590 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108a60 0x7fd9a4199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a4108f00 0x7fd9a419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 -- 172.21.15.112:0/3064809590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a4108f00 msgr2=0x7fd9a419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a4108f00 0x7fd9a419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 -- 172.21.15.112:0/3064809590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108a60 msgr2=0x7fd9a4199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108a60 0x7fd9a4199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a2ffd640 1 -- 172.21.15.112:0/3064809590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a419e4c0 con 0x7fd9a41031c0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.422+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108a60 0x7fd9a4199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.423+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a4199660 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd99400e9c0 tx=0x7fd99400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.423+0000 7fd9a88f7640 1 -- 172.21.15.112:0/3064809590 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99400ccc0 con 0x7fd9a41031c0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.423+0000 7fd9a88f7640 1 -- 172.21.15.112:0/3064809590 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9940054d0 con 0x7fd9a41031c0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.423+0000 7fd9a88f7640 1 -- 172.21.15.112:0/3064809590 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994010430 con 0x7fd9a41031c0 2024-04-25T16:46:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.423+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a419e7b0 con 0x7fd9a41031c0 2024-04-25T16:46:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.423+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a419ecb0 con 0x7fd9a41031c0 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.425+0000 7fd9a88f7640 1 -- 172.21.15.112:0/3064809590 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fd994005020 con 0x7fd9a41031c0 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.425+0000 7fd9b07ae740 1 Processor -- start 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.425+0000 7fd9b07ae740 1 -- start start 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117482f0 0x560411b12bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117504d0 0x560411b13150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560411a77460 con 0x5604117711b0 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560411a31590 con 0x5604117504d0 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560411a73420 con 0x5604117482f0 2024-04-25T16:46:21.432 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117482f0 0x560411b12bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b142a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33468/0 (socket says 172.21.15.112:33468) 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 -- 172.21.15.112:0/496778596 learned_addr learned my addr 172.21.15.112:0/496778596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117504d0 0x560411b13150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 -- 172.21.15.112:0/496778596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117482f0 msgr2=0x560411b12bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117482f0 0x560411b12bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a88f7640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd98c0afc50 0x7fd98c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 -- 172.21.15.112:0/496778596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117504d0 msgr2=0x560411b13150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117504d0 0x560411b13150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 -- 172.21.15.112:0/496778596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560411a3e0d0 con 0x5604117711b0 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117504d0 0x560411b13150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd98c0afc50 0x7fd98c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a88f7640 1 -- 172.21.15.112:0/3064809590 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7fd9940df3e0 con 0x7fd9a41031c0 2024-04-25T16:46:21.433 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.426+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b142a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd99c02f410 tx=0x7fd99c02f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150afd55b4d09419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.427+0000 7fd989ffb640 1 -- 172.21.15.112:0/496778596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c031db0 con 0x5604117711b0 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.427+0000 7fd989ffb640 1 -- 172.21.15.112:0/496778596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c036030 con 0x5604117711b0 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.427+0000 7fd989ffb640 1 -- 172.21.15.112:0/496778596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c02f660 con 0x5604117711b0 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 -- 172.21.15.112:0/496778596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 msgr2=0x560411b142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b142a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd99c02f410 tx=0x7fd99c02f440 comp rx=0 tx=0).stop 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd98c0afc50 0x7fd98c0b2160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd99c04a000 tx=0x7fd99c04b000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 -- 172.21.15.112:0/496778596 shutdown_connections 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117482f0 0x560411b12bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117504d0 0x560411b13150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/496778596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b142a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 -- 172.21.15.112:0/496778596 >> 172.21.15.112:0/496778596 conn(0x560411765300 msgr2=0x560411b0f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:21.434 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 -- 172.21.15.112:0/496778596 shutdown_connections 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 -- 172.21.15.112:0/496778596 wait complete. 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 -- start start 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.428+0000 7fd9b07ae740 1 Processor -- start 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b1f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117504d0 0x560411b1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b25330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560411b21f80 con 0x5604117711b0 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5604119feb60 con 0x5604117482f0 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560411b22100 con 0x5604117504d0 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9b07ae740 10 client.0 _ll_register_callbacks cb 0x560411a232b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b25330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b25330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33472/0 (socket says 172.21.15.112:33472) 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 -- 172.21.15.112:0/2428549797 learned_addr learned my addr 172.21.15.112:0/2428549797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117504d0 0x560411b1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b1f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 -- 172.21.15.112:0/2428549797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117504d0 msgr2=0x560411b1fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117504d0 0x560411b1fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 -- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 msgr2=0x560411b1f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b1f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 -- 172.21.15.112:0/2428549797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560411789210 con 0x5604117711b0 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b25330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006010 tx=0x7fd99c02f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a4e32be1ecfcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.429+0000 7fd982ffd640 1 -- 172.21.15.112:0/2428549797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c05f060 con 0x5604117711b0 2024-04-25T16:46:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c036030 con 0x5604117711b0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd982ffd640 1 -- 172.21.15.112:0/2428549797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7fd99c0046f0 con 0x5604117711b0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd982ffd640 1 -- 172.21.15.112:0/2428549797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd99c009360 con 0x5604117711b0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560411855620 con 0x5604117711b0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd982ffd640 1 -- 172.21.15.112:0/2428549797 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c036030 con 0x5604117711b0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd982ffd640 1 -- 172.21.15.112:0/2428549797 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fd99c003e80 con 0x5604117711b0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 20 client.16041 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56041172fd80 0x560411b31830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 4 client.16041 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560411a85070 con 0x56041172fd80 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.430+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56041172fd80 0x560411b31830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.431+0000 7fd982ffd640 10 client.16041 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.431+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56041172fd80 0x560411b31830 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17eb465ecf054ea6 server_cookie=edcc385dbde3e995 in_seq=0 out_seq=0 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd982ffd640 1 -- 172.21.15.112:0/2428549797 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560411a85070 con 0x56041172fd80 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd982ffd640 10 client.16041 handle_command_reply: tid=0 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 client.16041 shutdown 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 msgr2=0x560411b25330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b25330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd99c006010 tx=0x7fd99c02f890 comp rx=0 tx=0).stop 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 shutdown_connections 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56041172fd80 0x560411b31830 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117504d0 0x560411b1fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117711b0 0x560411b25330 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd982ffd640 0 client.16041 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:21.438 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2428549797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b1f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 >> 172.21.15.112:0/2428549797 conn(0x560411765300 msgr2=0x560411b153e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 shutdown_connections 2024-04-25T16:46:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2428549797 wait complete. 2024-04-25T16:46:21.439 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.432+0000 7fd9b07ae740 20 client.16041 trim_cache size 0 max 16384 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 Processor -- start 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 -- start start 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b0e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b25260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b21410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41b02d0 con 0x5604117504d0 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41adba0 con 0x5604117482f0 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41aba10 con 0x5604117711b0 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b25260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b25260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33488/0 (socket says 172.21.15.112:33488) 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 -- 172.21.15.112:0/1432957115 learned_addr learned my addr 172.21.15.112:0/1432957115 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b0e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b21410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 -- 172.21.15.112:0/1432957115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 msgr2=0x560411b21410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b21410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 -- 172.21.15.112:0/1432957115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 msgr2=0x560411b0e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b0e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.448+0000 7fd9a27fc640 1 -- 172.21.15.112:0/1432957115 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560411b134c0 con 0x5604117504d0 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.449+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b25260 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd998012610 tx=0x7fd998005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b9bec2605225cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.449+0000 7fd9897fa640 1 -- 172.21.15.112:0/1432957115 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998021070 con 0x5604117504d0 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.449+0000 7fd9897fa640 1 -- 172.21.15.112:0/1432957115 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9980035f0 con 0x5604117504d0 2024-04-25T16:46:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.449+0000 7fd9897fa640 1 -- 172.21.15.112:0/1432957115 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9980044e0 con 0x5604117504d0 2024-04-25T16:46:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:21 smithi133 ceph-mon[32691]: pgmap v246: 97 pgs: 97 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-04-25T16:46:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.449+0000 7fd9b07ae740 1 -- 172.21.15.112:0/1432957115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 msgr2=0x560411b25260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.449+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b25260 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd998012610 tx=0x7fd998005f90 comp rx=0 tx=0).stop 2024-04-25T16:46:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- 172.21.15.112:0/1432957115 shutdown_connections 2024-04-25T16:46:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b21410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b25260 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/1432957115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b0e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- 172.21.15.112:0/1432957115 >> 172.21.15.112:0/1432957115 conn(0x560411765300 msgr2=0x560411b20d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- 172.21.15.112:0/1432957115 shutdown_connections 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- 172.21.15.112:0/1432957115 wait complete. 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- start start 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 Processor -- start 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b29900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b29e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ac060 con 0x5604117504d0 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ad4d0 con 0x5604117482f0 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.450+0000 7fd9b07ae740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ac730 con 0x5604117711b0 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9b07ae740 10 client.0 _ll_register_callbacks cb 0x7fd99c049020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:21.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b29900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b29900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33504/0 (socket says 172.21.15.112:33504) 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 -- 172.21.15.112:0/2976108279 learned_addr learned my addr 172.21.15.112:0/2976108279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a37fe640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 -- 172.21.15.112:0/2976108279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 msgr2=0x560411b29e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b29e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 -- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 msgr2=0x560411b293c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b293c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 -- 172.21.15.112:0/2976108279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560411b989e0 con 0x5604117504d0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9a27fc640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b29900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd99801f040 tx=0x7fd998004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21cc28273dc693c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd982ffd640 1 -- 172.21.15.112:0/2976108279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99802b020 con 0x5604117504d0 2024-04-25T16:46:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd982ffd640 1 -- 172.21.15.112:0/2976108279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7fd998005050 con 0x5604117504d0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9980035f0 con 0x5604117504d0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd982ffd640 1 -- 172.21.15.112:0/2976108279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd998036050 con 0x5604117504d0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9b07ae740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.451+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x560411b99c50 con 0x5604117504d0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd982ffd640 1 -- 172.21.15.112:0/2976108279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9980035f0 con 0x5604117504d0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd982ffd640 1 -- 172.21.15.112:0/2976108279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7fd998004b70 con 0x5604117504d0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9b07ae740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9b07ae740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9b07ae740 20 client.16047 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5604117051c0 0x560411b9c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9b07ae740 4 client.16047 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560411a7eee0 con 0x5604117051c0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5604117051c0 0x560411b9c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd982ffd640 10 client.16047 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.452+0000 7fd9a2ffd640 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5604117051c0 0x560411b9c2d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c25bc6922527740e server_cookie=6a71b54974a4b92d in_seq=0 out_seq=0 2024-04-25T16:46:21.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:21 smithi116 ceph-mon[32712]: pgmap v246: 97 pgs: 97 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd982ffd640 1 -- 172.21.15.112:0/2976108279 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560411a7eee0 con 0x5604117051c0 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd982ffd640 10 client.16047 handle_command_reply: tid=0 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 client.16047 shutdown 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 msgr2=0x560411b29900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b29900 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd99801f040 tx=0x7fd998004260 comp rx=0 tx=0).stop 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 shutdown_connections 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5604117051c0 0x560411b9c2d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd982ffd640 0 client.16047 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5604117711b0 0x560411b29e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5604117504d0 0x560411b29900 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 --2- 172.21.15.112:0/2976108279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5604117482f0 0x560411b293c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 >> 172.21.15.112:0/2976108279 conn(0x560411765300 msgr2=0x560411b14900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 shutdown_connections 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.619+0000 7fd9b07ae740 1 -- 172.21.15.112:0/2976108279 wait complete. 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9b07ae740 20 client.16047 trim_cache size 0 max 16384 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:46:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:46:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd98c0afc50 msgr2=0x7fd98c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd98c0afc50 0x7fd98c0b2160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd99c04a000 tx=0x7fd99c04b000 comp rx=0 tx=0).stop 2024-04-25T16:46:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 msgr2=0x7fd9a4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:21.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a4199660 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd99400e9c0 tx=0x7fd99400ee80 comp rx=0 tx=0).stop 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 shutdown_connections 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fd98c0afc50 0x7fd98c0b2160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9a4108f00 0x7fd9a419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a4108a60 0x7fd9a4199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 --2- 172.21.15.112:0/3064809590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9a41031c0 0x7fd9a4199660 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.620+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 >> 172.21.15.112:0/3064809590 conn(0x7fd9a40781b0 msgr2=0x7fd9a4100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.621+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 shutdown_connections 2024-04-25T16:46:21.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:21.621+0000 7fd9a98f9640 1 -- 172.21.15.112:0/3064809590 wait complete. 2024-04-25T16:46:21.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 837714 10.94 MB/sec execute 267 sec latency 101.433 ms 2024-04-25T16:46:22.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 839426 10.94 MB/sec execute 268 sec latency 16.653 ms 2024-04-25T16:46:23.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:23 smithi112 ceph-mon[29122]: pgmap v247: 97 pgs: 97 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-04-25T16:46:23.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:23 smithi133 ceph-mon[32691]: pgmap v247: 97 pgs: 97 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-04-25T16:46:23.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:23 smithi116 ceph-mon[32712]: pgmap v247: 97 pgs: 97 active+clean; 80 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.9 MiB/s wr, 230 op/s 2024-04-25T16:46:23.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 841704 10.95 MB/sec execute 269 sec latency 17.504 ms 2024-04-25T16:46:24.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 843010 10.93 MB/sec execute 270 sec latency 84.647 ms 2024-04-25T16:46:25.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:25 smithi112 ceph-mon[29122]: pgmap v248: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 377 op/s 2024-04-25T16:46:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:25 smithi133 ceph-mon[32691]: pgmap v248: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 377 op/s 2024-04-25T16:46:25.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:25 smithi116 ceph-mon[32712]: pgmap v248: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.9 MiB/s wr, 377 op/s 2024-04-25T16:46:25.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 845285 10.94 MB/sec execute 271 sec latency 16.381 ms 2024-04-25T16:46:26.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 846999 10.92 MB/sec execute 272 sec latency 22.146 ms 2024-04-25T16:46:27.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:27 smithi112 ceph-mon[29122]: pgmap v249: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-04-25T16:46:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:27 smithi133 ceph-mon[32691]: pgmap v249: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-04-25T16:46:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:27 smithi116 ceph-mon[32712]: pgmap v249: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-04-25T16:46:27.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 849152 10.93 MB/sec execute 273 sec latency 14.251 ms 2024-04-25T16:46:28.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 851456 10.93 MB/sec execute 274 sec latency 16.491 ms 2024-04-25T16:46:29.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:29 smithi112 ceph-mon[29122]: pgmap v250: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-04-25T16:46:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:29 smithi133 ceph-mon[32691]: pgmap v250: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-04-25T16:46:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:29 smithi116 ceph-mon[32712]: pgmap v250: 97 pgs: 97 active+clean; 71 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.3 MiB/s wr, 283 op/s 2024-04-25T16:46:29.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 852775 10.92 MB/sec execute 275 sec latency 52.406 ms 2024-04-25T16:46:30.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 855093 10.92 MB/sec execute 276 sec latency 14.801 ms 2024-04-25T16:46:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:31 smithi133 ceph-mon[32691]: pgmap v251: 97 pgs: 97 active+clean; 77 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 378 op/s 2024-04-25T16:46:31.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:31 smithi116 ceph-mon[32712]: pgmap v251: 97 pgs: 97 active+clean; 77 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 378 op/s 2024-04-25T16:46:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:31 smithi112 ceph-mon[29122]: pgmap v251: 97 pgs: 97 active+clean; 77 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 378 op/s 2024-04-25T16:46:31.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 856833 10.92 MB/sec execute 277 sec latency 17.046 ms 2024-04-25T16:46:32.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 859306 10.93 MB/sec execute 278 sec latency 18.514 ms 2024-04-25T16:46:33.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:33 smithi133 ceph-mon[32691]: pgmap v252: 97 pgs: 97 active+clean; 77 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-04-25T16:46:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:33 smithi116 ceph-mon[32712]: pgmap v252: 97 pgs: 97 active+clean; 77 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-04-25T16:46:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:33 smithi112 ceph-mon[29122]: pgmap v252: 97 pgs: 97 active+clean; 77 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.0 MiB/s wr, 282 op/s 2024-04-25T16:46:33.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 860843 10.92 MB/sec execute 279 sec latency 16.466 ms 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 -- 172.21.15.112:0/3924653718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20104d30 msgr2=0x7f5f20105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 --2- 172.21.15.112:0/3924653718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20104d30 0x7f5f20105140 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009900 tx=0x7f5f1c02a480 comp rx=0 tx=0).stop 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 -- 172.21.15.112:0/3924653718 shutdown_connections 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 --2- 172.21.15.112:0/3924653718 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2010b790 0x7f5f2010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 --2- 172.21.15.112:0/3924653718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f2010e0f0 0x7f5f201105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 --2- 172.21.15.112:0/3924653718 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f20104d30 0x7f5f20105140 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 -- 172.21.15.112:0/3924653718 >> 172.21.15.112:0/3924653718 conn(0x7f5f200781b0 msgr2=0x7f5f200785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:34.737 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 -- 172.21.15.112:0/3924653718 shutdown_connections 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.736+0000 7f5f27940640 1 -- 172.21.15.112:0/3924653718 wait complete. 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 Processor -- start 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 -- start start 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f20104d30 0x7f5f2019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 0x7f5f2019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2010e0f0 0x7f5f2019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20113d30 con 0x7f5f2010b790 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20113bb0 con 0x7f5f20104d30 2024-04-25T16:46:34.738 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f27940640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f20113eb0 con 0x7f5f2010e0f0 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f24edb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 0x7f5f2019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f24edb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 0x7f5f2019acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41608/0 (socket says 172.21.15.112:41608) 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f24edb640 1 -- 172.21.15.112:0/3747470165 learned_addr learned my addr 172.21.15.112:0/3747470165 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.737+0000 7f5f256dc640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f20104d30 0x7f5f2019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2010e0f0 0x7f5f2019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f24edb640 1 -- 172.21.15.112:0/3747470165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2010e0f0 msgr2=0x7f5f2019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2010e0f0 0x7f5f2019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f24edb640 1 -- 172.21.15.112:0/3747470165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f20104d30 msgr2=0x7f5f2019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f20104d30 0x7f5f2019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f24edb640 1 -- 172.21.15.112:0/3747470165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2019f5f0 con 0x7f5f2010b790 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 0x7f5f2019acf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000e9f0 tx=0x7f5f1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f167fc640 1 -- 172.21.15.112:0/3747470165 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1000cd80 con 0x7f5f2010b790 2024-04-25T16:46:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f167fc640 1 -- 172.21.15.112:0/3747470165 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f100054d0 con 0x7f5f2010b790 2024-04-25T16:46:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2019f8e0 con 0x7f5f2010b790 2024-04-25T16:46:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f167fc640 1 -- 172.21.15.112:0/3747470165 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10010690 con 0x7f5f2010b790 2024-04-25T16:46:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.738+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2019fde0 con 0x7f5f2010b790 2024-04-25T16:46:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 Processor -- start 2024-04-25T16:46:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f167fc640 1 -- 172.21.15.112:0/3747470165 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f5f10004e90 con 0x7f5f2010b790 2024-04-25T16:46:34.748 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 -- start start 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a221f5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a221f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a2215b460 con 0x557a21e2c2f0 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557a22115590 con 0x557a21e551b0 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557a22157420 con 0x557a21e344d0 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f25edd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f256dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a221f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f25edd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:59578/0 (socket says 172.21.15.112:59578) 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f25edd640 1 -- 172.21.15.112:0/3678322680 learned_addr learned my addr 172.21.15.112:0/3678322680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.741+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a221f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f167fc640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5f0c0afc50 0x7f5f0c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.749 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 -- 172.21.15.112:0/3678322680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 msgr2=0x557a221f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a221f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 -- 172.21.15.112:0/3678322680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 msgr2=0x557a221f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a221f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 -- 172.21.15.112:0/3678322680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a221220d0 con 0x557a21e551b0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f167fc640 1 -- 172.21.15.112:0/3747470165 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f5f100df4a0 con 0x7f5f2010b790 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5f0c0afc50 0x7f5f0c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221f4af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009f90 tx=0x7f5f18010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bebb5e63038ed04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f0b7fe640 1 -- 172.21.15.112:0/3678322680 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1801f070 con 0x557a21e551b0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f0b7fe640 1 -- 172.21.15.112:0/3678322680 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18011b60 con 0x557a21e551b0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.742+0000 7f5f0b7fe640 1 -- 172.21.15.112:0/3678322680 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18018610 con 0x557a21e551b0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3678322680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 msgr2=0x557a221f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5f0c0afc50 0x7f5f0c0b2160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f18011010 tx=0x7f5f1801af90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:34.750 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221f4af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f18009f90 tx=0x7f5f18010860 comp rx=0 tx=0).stop 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3678322680 shutdown_connections 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a221f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a221f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3678322680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221f4af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3678322680 >> 172.21.15.112:0/3678322680 conn(0x557a21e49300 msgr2=0x557a221f12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3678322680 shutdown_connections 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.743+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3678322680 wait complete. 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 -- start start 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 Processor -- start 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a22200b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a22201050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221fefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557a22201c20 con 0x557a21e2c2f0 2024-04-25T16:46:34.751 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557a220e2b60 con 0x557a21e551b0 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557a22201da0 con 0x557a21e344d0 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f2e7f0740 10 client.0 _ll_register_callbacks cb 0x557a221072b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f256dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a22200b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f256dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a22200b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41628/0 (socket says 172.21.15.112:41628) 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f256dc640 1 -- 172.21.15.112:0/3107493799 learned_addr learned my addr 172.21.15.112:0/3107493799 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f25edd640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221fefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.744+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a22201050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f256dc640 1 -- 172.21.15.112:0/3107493799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 msgr2=0x557a22201050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f256dc640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a22201050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f256dc640 1 -- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 msgr2=0x557a221fefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f256dc640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221fefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f256dc640 1 -- 172.21.15.112:0/3107493799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a21e6d210 con 0x557a21e2c2f0 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f256dc640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a22200b10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009900 tx=0x7f5f1c0316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5840f203be9f6248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f08ff9640 1 -- 172.21.15.112:0/3107493799 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c035040 con 0x557a21e2c2f0 2024-04-25T16:46:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f18011b60 con 0x557a21e2c2f0 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f08ff9640 1 -- 172.21.15.112:0/3107493799 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f5f1c004040 con 0x557a21e2c2f0 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f08ff9640 1 -- 172.21.15.112:0/3107493799 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f1c032e30 con 0x557a21e2c2f0 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f2e7f0740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a21f39620 con 0x557a21e2c2f0 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f08ff9640 1 -- 172.21.15.112:0/3107493799 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18011b60 con 0x557a21e2c2f0 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.745+0000 7f5f08ff9640 1 -- 172.21.15.112:0/3107493799 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f5f1c004870 con 0x557a21e2c2f0 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f2e7f0740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f2e7f0740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f2e7f0740 20 client.16059 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21e13d80 0x557a222117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f2e7f0740 4 client.16059 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a22169070 con 0x557a21e13d80 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21e13d80 0x557a222117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f08ff9640 10 client.16059 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.746+0000 7f5f24edb640 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21e13d80 0x557a222117c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=431d31d1d6efe67f server_cookie=49f3c3b19d9bb843 in_seq=0 out_seq=0 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.747+0000 7f5f08ff9640 1 -- 172.21.15.112:0/3107493799 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a22169070 con 0x557a21e13d80 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.747+0000 7f5f08ff9640 10 client.16059 handle_command_reply: tid=0 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.747+0000 7f5f2e7f0740 1 client.16059 shutdown 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.747+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 msgr2=0x557a22200b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.747+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a22200b10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009900 tx=0x7f5f1c0316d0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 shutdown_connections 2024-04-25T16:46:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21e13d80 0x557a222117c0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e344d0 0x557a22201050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f08ff9640 0 client.16059 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a221fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/3107493799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e2c2f0 0x557a22200b10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 >> 172.21.15.112:0/3107493799 conn(0x557a21e49300 msgr2=0x557a221f4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 shutdown_connections 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/3107493799 wait complete. 2024-04-25T16:46:34.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.748+0000 7f5f2e7f0740 20 client.16059 trim_cache size 0 max 16384 2024-04-25T16:46:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 Processor -- start 2024-04-25T16:46:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 -- start start 2024-04-25T16:46:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a221fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e344d0 0x557a22207030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 0x557a22207570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201b2430 con 0x557a21e551b0 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201afd00 con 0x557a21e344d0 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201adb70 con 0x557a21e2c2f0 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f25edd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 0x557a22207570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f25edd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 0x557a22207570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41636/0 (socket says 172.21.15.112:41636) 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f24edb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e344d0 0x557a22207030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f25edd640 1 -- 172.21.15.112:0/2665036425 learned_addr learned my addr 172.21.15.112:0/2665036425 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.763+0000 7f5f256dc640 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a221fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f25edd640 1 -- 172.21.15.112:0/2665036425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 msgr2=0x557a221fef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f25edd640 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a221fef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f25edd640 1 -- 172.21.15.112:0/2665036425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e344d0 msgr2=0x557a22207030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f25edd640 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e344d0 0x557a22207030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f25edd640 1 -- 172.21.15.112:0/2665036425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a22207ab0 con 0x557a21e551b0 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f256dc640 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a221fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f24edb640 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e344d0 0x557a22207030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f25edd640 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 0x557a22207570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x557a221f1c60 tx=0x7f5f18027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a38ccfb25515658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f0affd640 1 -- 172.21.15.112:0/2665036425 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18004a00 con 0x557a21e551b0 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f0affd640 1 -- 172.21.15.112:0/2665036425 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18035070 con 0x557a21e551b0 2024-04-25T16:46:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.764+0000 7f5f0affd640 1 -- 172.21.15.112:0/2665036425 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f18011860 con 0x557a21e551b0 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2665036425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 msgr2=0x557a22207570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 0x557a22207570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x557a221f1c60 tx=0x7f5f18027000 comp rx=0 tx=0).stop 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2665036425 shutdown_connections 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e551b0 0x557a22207570 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e344d0 0x557a22207030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2665036425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a221fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2665036425 >> 172.21.15.112:0/2665036425 conn(0x557a21e49300 msgr2=0x557a221fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2665036425 shutdown_connections 2024-04-25T16:46:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2665036425 wait complete. 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 -- start start 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.765+0000 7f5f2e7f0740 1 Processor -- start 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a22272560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 0x557a22272aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a222068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201ae1c0 con 0x557a21e344d0 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201af630 con 0x557a21e551b0 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f201ae890 con 0x557a21e2c2f0 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 10 client.0 _ll_register_callbacks cb 0x7f5f1c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 0x557a22272aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 0x557a22272aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41648/0 (socket says 172.21.15.112:41648) 2024-04-25T16:46:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 -- 172.21.15.112:0/2661901916 learned_addr learned my addr 172.21.15.112:0/2661901916 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f25edd640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a222068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f256dc640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a22272560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 -- 172.21.15.112:0/2661901916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 msgr2=0x557a22272560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a22272560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 -- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 msgr2=0x557a222068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a222068f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 -- 172.21.15.112:0/2661901916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a22206f00 con 0x557a21e344d0 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f24edb640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 0x557a22272aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c6a0 tx=0x7f5f100e5060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c552fa9a805defc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f08ff9640 1 -- 172.21.15.112:0/2661901916 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f100a6d60 con 0x557a21e344d0 2024-04-25T16:46:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f08ff9640 1 -- 172.21.15.112:0/2661901916 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f5f100a3880 con 0x557a21e344d0 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.766+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f18035070 con 0x557a21e344d0 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f08ff9640 1 -- 172.21.15.112:0/2661901916 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5f10005180 con 0x557a21e344d0 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 10 client.16065 fetch_fsmap learned FSMap version 27 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557a22279c70 con 0x557a21e344d0 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f08ff9640 1 -- 172.21.15.112:0/2661901916 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f18035070 con 0x557a21e344d0 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f08ff9640 1 -- 172.21.15.112:0/2661901916 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 27) ==== 8338+0+0 (secure 0 0 0) 0x7f5f100de040 con 0x557a21e344d0 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 20 client.16065 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21de91c0 0x557a2227c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 4 client.16065 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:46:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557a22162ee0 con 0x557a21de91c0 2024-04-25T16:46:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f256dc640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21de91c0 0x557a2227c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f08ff9640 10 client.16065 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.767+0000 7f5f256dc640 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21de91c0 0x557a2227c3e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4113c00deafdb5c0 server_cookie=e44eee4662c26e8e in_seq=0 out_seq=0 2024-04-25T16:46:34.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 862824 10.92 MB/sec execute 280 sec latency 70.074 ms 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f08ff9640 1 -- 172.21.15.112:0/2661901916 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557a22162ee0 con 0x557a21de91c0 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f08ff9640 10 client.16065 handle_command_reply: tid=0 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 client.16065 shutdown 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 msgr2=0x557a22272aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 0x557a22272aa0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000c6a0 tx=0x7f5f100e5060 comp rx=0 tx=0).stop 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 shutdown_connections 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557a21e551b0 0x557a222068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557a21e344d0 0x557a22272aa0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557a21de91c0 0x557a2227c3e0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.924 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 --2- 172.21.15.112:0/2661901916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a21e2c2f0 0x557a22272560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 >> 172.21.15.112:0/2661901916 conn(0x557a21e49300 msgr2=0x557a221f4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f08ff9640 0 client.16065 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 shutdown_connections 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 1 -- 172.21.15.112:0/2661901916 wait complete. 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.925+0000 7f5f2e7f0740 20 client.16065 trim_cache size 0 max 16384 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:46:34.925 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5f0c0afc50 msgr2=0x7f5f0c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5f0c0afc50 0x7f5f0c0b2160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5f18011010 tx=0x7f5f1801af90 comp rx=0 tx=0).stop 2024-04-25T16:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 msgr2=0x7f5f2019acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 0x7f5f2019acf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5f1000e9f0 tx=0x7f5f1000edf0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 shutdown_connections 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f5f0c0afc50 0x7f5f0c0b2160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f2010e0f0 0x7f5f2019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5f2010b790 0x7f5f2019acf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 --2- 172.21.15.112:0/3747470165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f20104d30 0x7f5f2019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 >> 172.21.15.112:0/3747470165 conn(0x7f5f200781b0 msgr2=0x7f5f20101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 shutdown_connections 2024-04-25T16:46:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:34.926+0000 7f5f27940640 1 -- 172.21.15.112:0/3747470165 wait complete. 2024-04-25T16:46:35.397 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:35 smithi112 ceph-mon[29122]: pgmap v253: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.3 MiB/s wr, 371 op/s 2024-04-25T16:46:35.424 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:35 smithi116 ceph-mon[32712]: pgmap v253: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.3 MiB/s wr, 371 op/s 2024-04-25T16:46:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:35 smithi133 ceph-mon[32691]: pgmap v253: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.3 MiB/s wr, 371 op/s 2024-04-25T16:46:35.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:35 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setuser ceph since I am not root 2024-04-25T16:46:35.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:35 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: ignoring --setgroup ceph since I am not root 2024-04-25T16:46:35.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:35 smithi116 ceph-mds[62762]: -- 172.21.15.116:0/1480109878 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563606705a40 con 0x56360674d180 2024-04-25T16:46:35.758 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:46:35 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: starting mds.e at 2024-04-25T16:46:35.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 864274 10.91 MB/sec execute 281 sec latency 85.673 ms 2024-04-25T16:46:36.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:36 smithi133 ceph-mon[32691]: daemon mds.e restarted 2024-04-25T16:46:36.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:36 smithi116 ceph-mon[32712]: daemon mds.e restarted 2024-04-25T16:46:36.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:36 smithi112 ceph-mon[29122]: daemon mds.e restarted 2024-04-25T16:46:36.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 866557 10.92 MB/sec execute 282 sec latency 17.724 ms 2024-04-25T16:46:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:37 smithi133 ceph-mon[32691]: pgmap v254: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-25T16:46:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:37 smithi133 ceph-mon[32691]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T16:46:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:37 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6834/1317678143,v1:172.21.15.116:6835/1317678143] up:boot 2024-04-25T16:46:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:37 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:46:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:37 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:46:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:37 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:46:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:37 smithi116 ceph-mon[32712]: pgmap v254: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-25T16:46:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:37 smithi116 ceph-mon[32712]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T16:46:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:37 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6834/1317678143,v1:172.21.15.116:6835/1317678143] up:boot 2024-04-25T16:46:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:37 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:46:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:37 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:46:37.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:37 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:46:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:37 smithi112 ceph-mon[29122]: pgmap v254: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 203 op/s 2024-04-25T16:46:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:37 smithi112 ceph-mon[29122]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T16:46:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:37 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6834/1317678143,v1:172.21.15.116:6835/1317678143] up:boot 2024-04-25T16:46:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:37 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:46:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:37 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T16:46:37.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:37 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:46:37.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 868208 10.90 MB/sec execute 283 sec latency 16.501 ms 2024-04-25T16:46:38.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 870393 10.91 MB/sec execute 284 sec latency 14.884 ms 2024-04-25T16:46:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:39 smithi133 ceph-mon[32691]: pgmap v256: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-04-25T16:46:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:39 smithi116 ceph-mon[32712]: pgmap v256: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-04-25T16:46:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:39 smithi112 ceph-mon[29122]: pgmap v256: 97 pgs: 97 active+clean; 84 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.5 MiB/s wr, 224 op/s 2024-04-25T16:46:39.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 872221 10.89 MB/sec execute 285 sec latency 88.502 ms 2024-04-25T16:46:40.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:40 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:46:40.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:40 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:46:40.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:40 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:46:40.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 874004 10.90 MB/sec execute 286 sec latency 18.658 ms 2024-04-25T16:46:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:41 smithi133 ceph-mon[32691]: pgmap v257: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-04-25T16:46:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:41 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:46:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:41 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:46:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:41 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:46:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:41 smithi116 ceph-mon[32712]: pgmap v257: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-04-25T16:46:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:41 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:46:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:41 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:46:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:41 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:46:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:41 smithi112 ceph-mon[29122]: pgmap v257: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-04-25T16:46:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:41 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:46:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:41 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:46:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:41 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:46:41.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 876304 10.90 MB/sec execute 287 sec latency 17.458 ms 2024-04-25T16:46:42.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 878112 10.90 MB/sec execute 288 sec latency 18.109 ms 2024-04-25T16:46:43.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:43 smithi133 ceph-mon[32691]: pgmap v258: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-04-25T16:46:43.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:43 smithi116 ceph-mon[32712]: pgmap v258: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-04-25T16:46:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:43 smithi112 ceph-mon[29122]: pgmap v258: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-04-25T16:46:43.885 INFO:tasks.workunit.client.0.smithi112.stdout: 1 880612 10.91 MB/sec execute 289 sec latency 17.518 ms 2024-04-25T16:46:44.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 882144 10.90 MB/sec execute 290 sec latency 117.640 ms 2024-04-25T16:46:45.450 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:45 smithi133 ceph-mon[32691]: pgmap v259: 97 pgs: 97 active+clean; 75 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2024-04-25T16:46:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:45 smithi116 ceph-mon[32712]: pgmap v259: 97 pgs: 97 active+clean; 75 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2024-04-25T16:46:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:45 smithi112 ceph-mon[29122]: pgmap v259: 97 pgs: 97 active+clean; 75 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 295 op/s 2024-04-25T16:46:45.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 884405 10.91 MB/sec execute 291 sec latency 33.542 ms 2024-04-25T16:46:46.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 886443 10.90 MB/sec execute 292 sec latency 20.201 ms 2024-04-25T16:46:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:47 smithi133 ceph-mon[32691]: pgmap v260: 97 pgs: 97 active+clean; 78 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:46:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:47 smithi116 ceph-mon[32712]: pgmap v260: 97 pgs: 97 active+clean; 78 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:46:47.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:47 smithi112 ceph-mon[29122]: pgmap v260: 97 pgs: 97 active+clean; 78 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.0 MiB/s wr, 320 op/s 2024-04-25T16:46:47.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 888480 10.90 MB/sec execute 293 sec latency 17.477 ms 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 -- 172.21.15.112:0/1050234260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f410e5a0 msgr2=0x7f92f41109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 --2- 172.21.15.112:0/1050234260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f410e5a0 0x7f92f41109c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009ca0 tx=0x7f92ec02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 -- 172.21.15.112:0/1050234260 shutdown_connections 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 --2- 172.21.15.112:0/1050234260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f41020a0 0x7f92f41024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 --2- 172.21.15.112:0/1050234260 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f410e5a0 0x7f92f41109c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 --2- 172.21.15.112:0/1050234260 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f4109bf0 0x7f92f4101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 -- 172.21.15.112:0/1050234260 >> 172.21.15.112:0/1050234260 conn(0x7f92f40778b0 msgr2=0x7f92f4077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 -- 172.21.15.112:0/1050234260 shutdown_connections 2024-04-25T16:46:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.028+0000 7f92f99d7640 1 -- 172.21.15.112:0/1050234260 wait complete. 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 Processor -- start 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 -- start start 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 0x7f92f419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4109bf0 0x7f92f419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f410e5a0 0x7f92f419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4113d60 con 0x7f92f4109bf0 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4113be0 con 0x7f92f41020a0 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.029+0000 7f92f99d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92f4113ee0 con 0x7f92f410e5a0 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 0x7f92f419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4109bf0 0x7f92f419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 0x7f92f419a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:59554/0 (socket says 172.21.15.112:59554) 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 -- 172.21.15.112:0/3927936125 learned_addr learned my addr 172.21.15.112:0/3927936125 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f37fe640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f410e5a0 0x7f92f419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 -- 172.21.15.112:0/3927936125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f410e5a0 msgr2=0x7f92f419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f410e5a0 0x7f92f419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 -- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4109bf0 msgr2=0x7f92f419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4109bf0 0x7f92f419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 -- 172.21.15.112:0/3927936125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f419f5a0 con 0x7f92f41020a0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 0x7f92f419a740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92e400e9f0 tx=0x7f92e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f89d5640 1 -- 172.21.15.112:0/3927936125 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400ccc0 con 0x7f92f41020a0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f419f890 con 0x7f92f41020a0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f89d5640 1 -- 172.21.15.112:0/3927936125 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e40035f0 con 0x7f92f41020a0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f89d5640 1 -- 172.21.15.112:0/3927936125 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e400ce60 con 0x7f92f41020a0 2024-04-25T16:46:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.030+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41a7210 con 0x7f92f41020a0 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 Processor -- start 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 -- start start 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f92f89d5640 1 -- 172.21.15.112:0/3927936125 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f92e4003aa0 con 0x7f92f41020a0 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 0x556b265689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b26568ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b26569420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556b264cf460 con 0x556b261c91b0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556b26489590 con 0x556b261a02f0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f9300964740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b264cb420 con 0x556b261a84d0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 0x556b265689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 0x556b265689a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40442/0 (socket says 172.21.15.112:40442) 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f92f37fe640 1 -- 172.21.15.112:0/147174886 learned_addr learned my addr 172.21.15.112:0/147174886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b26568ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.033+0000 7f92f27fc640 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b26569420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f37fe640 1 -- 172.21.15.112:0/147174886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 msgr2=0x556b26569420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f37fe640 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b26569420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f37fe640 1 -- 172.21.15.112:0/147174886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 msgr2=0x556b26568ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f37fe640 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b26568ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f37fe640 1 -- 172.21.15.112:0/147174886 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b264960d0 con 0x556b261c91b0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f37fe640 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 0x556b265689a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f92ec035000 tx=0x7f92ec02f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8616c035e6f1450c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f89d5640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f92dc0afc50 0x7f92dc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f92dc0afc50 0x7f92dc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92d9ffb640 1 -- 172.21.15.112:0/147174886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec031cf0 con 0x556b261c91b0 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92d9ffb640 1 -- 172.21.15.112:0/147174886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ec036030 con 0x556b261c91b0 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92d9ffb640 1 -- 172.21.15.112:0/147174886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec004040 con 0x556b261c91b0 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.034+0000 7f92f89d5640 1 -- 172.21.15.112:0/3927936125 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8556+0+0 (secure 0 0 0) 0x7f92e40df2f0 con 0x7f92f41020a0 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 -- 172.21.15.112:0/147174886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 msgr2=0x556b265689a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f92f2ffd640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f92dc0afc50 0x7f92dc0b2160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92e40dcad0 tx=0x7f92e4090040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 0x556b265689a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f92ec035000 tx=0x7f92ec02f410 comp rx=0 tx=0).stop 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 -- 172.21.15.112:0/147174886 shutdown_connections 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b26569420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b26568ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 --2- 172.21.15.112:0/147174886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261c91b0 0x556b265689a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 -- 172.21.15.112:0/147174886 >> 172.21.15.112:0/147174886 conn(0x556b261bd300 msgr2=0x556b265652a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 -- 172.21.15.112:0/147174886 shutdown_connections 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 -- 172.21.15.112:0/147174886 wait complete. 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.035+0000 7f9300964740 1 -- start start 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 Processor -- start 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a02f0 0x556b265775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b26577b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b2657d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556b26579e50 con 0x556b261a84d0 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556b26456b60 con 0x556b261c91b0 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b26579fd0 con 0x556b261a02f0 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f9300964740 10 client.0 _ll_register_callbacks cb 0x556b2647b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b26577b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a02f0 0x556b265775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b26577b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40448/0 (socket says 172.21.15.112:40448) 2024-04-25T16:46:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f27fc640 1 -- 172.21.15.112:0/221187742 learned_addr learned my addr 172.21.15.112:0/221187742 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b2657d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 -- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 msgr2=0x556b2657d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b2657d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 -- 172.21.15.112:0/221187742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 msgr2=0x556b26577b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b26577b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.036+0000 7f92f37fe640 1 -- 172.21.15.112:0/221187742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b261e1210 con 0x556b261a02f0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92f37fe640 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a02f0 0x556b265775c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02f480 tx=0x7f92ec003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f65b2f57e0b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92d2ffd640 1 -- 172.21.15.112:0/221187742 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec059060 con 0x556b261a02f0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92d2ffd640 1 -- 172.21.15.112:0/221187742 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8556+0+0 (secure 0 0 0) 0x7f92ec0040b0 con 0x556b261a02f0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ec036030 con 0x556b261a02f0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92d2ffd640 1 -- 172.21.15.112:0/221187742 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92ec031920 con 0x556b261a02f0 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 10 client.25711 fetch_fsmap learned FSMap version 29 2024-04-25T16:46:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b262ad620 con 0x556b261a02f0 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92d2ffd640 1 -- 172.21.15.112:0/221187742 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ec036030 con 0x556b261a02f0 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f92d2ffd640 1 -- 172.21.15.112:0/221187742 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f92ec043070 con 0x556b261a02f0 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 10 client.25711 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 20 client.25711 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b26187d80 0x556b26589780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 4 client.25711 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.037+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b264dd070 con 0x556b26187d80 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.038+0000 7f92f27fc640 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b26187d80 0x556b26589780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.038+0000 7f92d2ffd640 10 client.25711 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.038+0000 7f92f27fc640 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b26187d80 0x556b26589780 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc18e8ef1e9c3d4 server_cookie=637e07dc407587e4 in_seq=0 out_seq=0 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f92d2ffd640 1 -- 172.21.15.112:0/221187742 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b264dd070 con 0x556b26187d80 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f92d2ffd640 10 client.25711 handle_command_reply: tid=0 2024-04-25T16:46:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 client.25711 shutdown 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a02f0 msgr2=0x556b265775c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a02f0 0x556b265775c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02f480 tx=0x7f92ec003b90 comp rx=0 tx=0).stop 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 shutdown_connections 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b2657d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b26577b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b26187d80 0x556b26589780 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 --2- 172.21.15.112:0/221187742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a02f0 0x556b265775c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 >> 172.21.15.112:0/221187742 conn(0x556b261bd300 msgr2=0x556b2656a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f92d2ffd640 0 client.25711 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 shutdown_connections 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 1 -- 172.21.15.112:0/221187742 wait complete. 2024-04-25T16:46:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.039+0000 7f9300964740 20 client.25711 trim_cache size 0 max 16384 2024-04-25T16:46:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.054+0000 7f9300964740 1 Processor -- start 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.054+0000 7f9300964740 1 -- start start 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.054+0000 7f9300964740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b2657d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.054+0000 7f9300964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b265762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.054+0000 7f9300964740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261c91b0 0x556b26570d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f9300964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41b0140 con 0x556b261a84d0 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f9300964740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ada10 con 0x556b261a02f0 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f9300964740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ab860 con 0x556b261c91b0 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b265762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b265762f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40450/0 (socket says 172.21.15.112:40450) 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 -- 172.21.15.112:0/1129297131 learned_addr learned my addr 172.21.15.112:0/1129297131 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f37fe640 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b2657d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f37fe640 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261c91b0 0x556b26570d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 -- 172.21.15.112:0/1129297131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261c91b0 msgr2=0x556b26570d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261c91b0 0x556b26570d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 -- 172.21.15.112:0/1129297131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 msgr2=0x556b2657d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b2657d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 -- 172.21.15.112:0/1129297131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b26565e90 con 0x556b261a84d0 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92f27fc640 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b265762f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x556b2657a0e0 tx=0x7f92e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26349833f5174585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92d97fa640 1 -- 172.21.15.112:0/1129297131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e8010430 con 0x556b261a84d0 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92d97fa640 1 -- 172.21.15.112:0/1129297131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8010a80 con 0x556b261a84d0 2024-04-25T16:46:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.055+0000 7f92d97fa640 1 -- 172.21.15.112:0/1129297131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e801ea00 con 0x556b261a84d0 2024-04-25T16:46:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 -- 172.21.15.112:0/1129297131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 msgr2=0x556b265762f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b265762f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x556b2657a0e0 tx=0x7f92e8005f90 comp rx=0 tx=0).stop 2024-04-25T16:46:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 -- 172.21.15.112:0/1129297131 shutdown_connections 2024-04-25T16:46:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261c91b0 0x556b26570d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a84d0 0x556b265762f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 --2- 172.21.15.112:0/1129297131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261a02f0 0x556b2657d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 -- 172.21.15.112:0/1129297131 >> 172.21.15.112:0/1129297131 conn(0x556b261bd300 msgr2=0x556b265692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 -- 172.21.15.112:0/1129297131 shutdown_connections 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.056+0000 7f9300964740 1 -- 172.21.15.112:0/1129297131 wait complete. 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 -- start start 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 Processor -- start 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 0x556b26583e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b265811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b26581710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41abed0 con 0x556b261a02f0 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ad340 con 0x556b261c91b0 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41ac5a0 con 0x556b261a84d0 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f9300964740 10 client.0 _ll_register_callbacks cb 0x7f92e40e2d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:46:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 0x556b26583e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f92f37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 0x556b26583e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40458/0 (socket says 172.21.15.112:40458) 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f92f37fe640 1 -- 172.21.15.112:0/1662978020 learned_addr learned my addr 172.21.15.112:0/1662978020 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f92f37fe640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b26581710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.057+0000 7f92f27fc640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b265811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92f37fe640 1 -- 172.21.15.112:0/1662978020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 msgr2=0x556b265811d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92f37fe640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b265811d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92f37fe640 1 -- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 msgr2=0x556b26581710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92f37fe640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b26581710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92f37fe640 1 -- 172.21.15.112:0/1662978020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b26581c50 con 0x556b261a02f0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92f37fe640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 0x556b26583e60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02ade0 tx=0x7f92ec003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b847ced18399c746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92d2ffd640 1 -- 172.21.15.112:0/1662978020 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec062040 con 0x556b261a02f0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92d2ffd640 1 -- 172.21.15.112:0/1662978020 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8556+0+0 (secure 0 0 0) 0x7f92ec004250 con 0x556b261a02f0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92e8010a80 con 0x556b261a02f0 2024-04-25T16:46:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92d2ffd640 1 -- 172.21.15.112:0/1662978020 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92ec059040 con 0x556b261a02f0 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b265f14c0 con 0x556b261a02f0 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92d2ffd640 1 -- 172.21.15.112:0/1662978020 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92e8010a80 con 0x556b261a02f0 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f92d2ffd640 1 -- 172.21.15.112:0/1662978020 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f92ec043070 con 0x556b261a02f0 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 10 client.16089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 20 client.16089 populate_metadata read hostname 'smithi112' 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b2615d1c0 0x556b265f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.058+0000 7f9300964740 4 client.16089 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.059+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b264d6ee0 con 0x556b2615d1c0 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.059+0000 7f92f27fc640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b2615d1c0 0x556b265f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:46:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.059+0000 7f92d2ffd640 10 client.16089 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.059+0000 7f92f27fc640 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b2615d1c0 0x556b265f3b00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca02f5a7e8935a44 server_cookie=1e58f40a4a984b51 in_seq=0 out_seq=0 2024-04-25T16:46:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.231+0000 7f92d2ffd640 1 -- 172.21.15.112:0/1662978020 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b264d6ee0 con 0x556b2615d1c0 2024-04-25T16:46:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.231+0000 7f92d2ffd640 10 client.16089 handle_command_reply: tid=0 2024-04-25T16:46:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.231+0000 7f9300964740 1 client.16089 shutdown 2024-04-25T16:46:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 msgr2=0x556b26583e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.231 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 0x556b26583e60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f92ec02ade0 tx=0x7f92ec003af0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 shutdown_connections 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556b2615d1c0 0x556b265f3b00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f92d2ffd640 0 client.16089 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b261a84d0 0x556b265811d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556b261c91b0 0x556b26581710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 --2- 172.21.15.112:0/1662978020 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556b261a02f0 0x556b26583e60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.232+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 >> 172.21.15.112:0/1662978020 conn(0x556b261bd300 msgr2=0x556b265792b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.233+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 shutdown_connections 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.233+0000 7f9300964740 1 -- 172.21.15.112:0/1662978020 wait complete. 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.233+0000 7f9300964740 20 client.16089 trim_cache size 0 max 16384 2024-04-25T16:46:48.232 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:46:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:46:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:46:48.233 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f92dc0afc50 msgr2=0x7f92dc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f92dc0afc50 0x7f92dc0b2160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f92e40dcad0 tx=0x7f92e4090040 comp rx=0 tx=0).stop 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 msgr2=0x7f92f419a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 0x7f92f419a740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f92e400e9f0 tx=0x7f92e400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 shutdown_connections 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f92dc0afc50 0x7f92dc0b2160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f410e5a0 0x7f92f419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92f4109bf0 0x7f92f419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.234 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 --2- 172.21.15.112:0/3927936125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92f41020a0 0x7f92f419a740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:46:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.234+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 >> 172.21.15.112:0/3927936125 conn(0x7f92f40778b0 msgr2=0x7f92f410a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:46:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.235+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 shutdown_connections 2024-04-25T16:46:48.235 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:46:48.235+0000 7f92f99d7640 1 -- 172.21.15.112:0/3927936125 wait complete. 2024-04-25T16:46:48.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 890563 10.91 MB/sec execute 294 sec latency 90.233 ms 2024-04-25T16:46:49.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:49 smithi133 ceph-mon[32691]: pgmap v261: 97 pgs: 97 active+clean; 78 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-04-25T16:46:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:49 smithi116 ceph-mon[32712]: pgmap v261: 97 pgs: 97 active+clean; 78 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-04-25T16:46:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:49 smithi112 ceph-mon[29122]: pgmap v261: 97 pgs: 97 active+clean; 78 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.6 MiB/s wr, 277 op/s 2024-04-25T16:46:49.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 892279 10.90 MB/sec execute 295 sec latency 13.884 ms 2024-04-25T16:46:50.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 894485 10.90 MB/sec execute 296 sec latency 53.792 ms 2024-04-25T16:46:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:51 smithi133 ceph-mon[32691]: pgmap v262: 97 pgs: 97 active+clean; 69 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-04-25T16:46:51.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:51 smithi116 ceph-mon[32712]: pgmap v262: 97 pgs: 97 active+clean; 69 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-04-25T16:46:51.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:51 smithi112 ceph-mon[29122]: pgmap v262: 97 pgs: 97 active+clean; 69 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.1 MiB/s wr, 456 op/s 2024-04-25T16:46:51.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 896219 10.90 MB/sec execute 297 sec latency 20.784 ms 2024-04-25T16:46:52.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 898591 10.91 MB/sec execute 298 sec latency 15.711 ms 2024-04-25T16:46:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:53 smithi133 ceph-mon[32691]: pgmap v263: 97 pgs: 97 active+clean; 69 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-04-25T16:46:53.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:53 smithi116 ceph-mon[32712]: pgmap v263: 97 pgs: 97 active+clean; 69 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-04-25T16:46:53.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:53 smithi112 ceph-mon[29122]: pgmap v263: 97 pgs: 97 active+clean; 69 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 276 op/s 2024-04-25T16:46:53.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 900535 10.90 MB/sec execute 299 sec latency 48.476 ms 2024-04-25T16:46:54.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 902551 10.90 MB/sec execute 300 sec latency 14.299 ms 2024-04-25T16:46:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:55 smithi133 ceph-mon[32691]: pgmap v264: 97 pgs: 97 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 369 op/s 2024-04-25T16:46:55.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:55 smithi116 ceph-mon[32712]: pgmap v264: 97 pgs: 97 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 369 op/s 2024-04-25T16:46:55.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:55 smithi112 ceph-mon[29122]: pgmap v264: 97 pgs: 97 active+clean; 76 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.1 MiB/s wr, 369 op/s 2024-04-25T16:46:55.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 904676 10.91 MB/sec execute 301 sec latency 44.264 ms 2024-04-25T16:46:56.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 906528 10.90 MB/sec execute 302 sec latency 13.559 ms 2024-04-25T16:46:57.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:57 smithi133 ceph-mon[32691]: pgmap v265: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-04-25T16:46:57.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:57 smithi116 ceph-mon[32712]: pgmap v265: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-04-25T16:46:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:57 smithi112 ceph-mon[29122]: pgmap v265: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-04-25T16:46:57.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 909050 10.92 MB/sec execute 303 sec latency 15.119 ms 2024-04-25T16:46:58.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 910543 10.90 MB/sec execute 304 sec latency 37.860 ms 2024-04-25T16:46:59.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:46:59 smithi133 ceph-mon[32691]: pgmap v266: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 289 op/s 2024-04-25T16:46:59.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:46:59 smithi116 ceph-mon[32712]: pgmap v266: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 289 op/s 2024-04-25T16:46:59.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:46:59 smithi112 ceph-mon[29122]: pgmap v266: 97 pgs: 97 active+clean; 76 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 289 op/s 2024-04-25T16:46:59.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 912819 10.91 MB/sec execute 305 sec latency 27.682 ms 2024-04-25T16:47:00.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 914243 10.89 MB/sec execute 306 sec latency 85.731 ms 2024-04-25T16:47:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.335+0000 7f9ed412e640 1 -- 172.21.15.112:0/1073406559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 msgr2=0x7f9ecc10fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.335+0000 7f9ed412e640 1 --2- 172.21.15.112:0/1073406559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc10fe90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc009900 tx=0x7f9ebc02a480 comp rx=0 tx=0).stop 2024-04-25T16:47:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 -- 172.21.15.112:0/1073406559 shutdown_connections 2024-04-25T16:47:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 --2- 172.21.15.112:0/1073406559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ecc1103d0 0x7f9ecc1127f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 --2- 172.21.15.112:0/1073406559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc10fe90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 --2- 172.21.15.112:0/1073406559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ecc0763f0 0x7f9ecc076800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 -- 172.21.15.112:0/1073406559 >> 172.21.15.112:0/1073406559 conn(0x7f9ecc0ff1e0 msgr2=0x7f9ecc101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 -- 172.21.15.112:0/1073406559 shutdown_connections 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 -- 172.21.15.112:0/1073406559 wait complete. 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 Processor -- start 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.336+0000 7f9ed412e640 1 -- start start 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed412e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ecc0763f0 0x7f9ecc1a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed412e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc1a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed412e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ecc1103d0 0x7f9ecc19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed412e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc115f40 con 0x7f9ecc10da70 2024-04-25T16:47:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed412e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc115dc0 con 0x7f9ecc1103d0 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed412e640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1160c0 con 0x7f9ecc0763f0 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc1a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc1a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42030/0 (socket says 172.21.15.112:42030) 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 -- 172.21.15.112:0/2854353597 learned_addr learned my addr 172.21.15.112:0/2854353597 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ecc1103d0 0x7f9ecc19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ecc0763f0 0x7f9ecc1a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 -- 172.21.15.112:0/2854353597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ecc0763f0 msgr2=0x7f9ecc1a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ecc0763f0 0x7f9ecc1a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 -- 172.21.15.112:0/2854353597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ecc1103d0 msgr2=0x7f9ecc19c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ecc1103d0 0x7f9ecc19c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 -- 172.21.15.112:0/2854353597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc19cec0 con 0x7f9ecc10da70 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.337+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc1a2af0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005ab0 tx=0x7f9ebc031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.338+0000 7f9ec2ffd640 1 -- 172.21.15.112:0/2854353597 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc03f030 con 0x7f9ecc10da70 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.338+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc19d150 con 0x7f9ecc10da70 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.338+0000 7f9ec2ffd640 1 -- 172.21.15.112:0/2854353597 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ebc0043d0 con 0x7f9ecc10da70 2024-04-25T16:47:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.338+0000 7f9ec2ffd640 1 -- 172.21.15.112:0/2854353597 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc035070 con 0x7f9ecc10da70 2024-04-25T16:47:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.338+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc1a9400 con 0x7f9ecc10da70 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.340+0000 7f9edafda740 1 Processor -- start 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 -- start start 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5dcaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5dcb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dcb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9a5d31460 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e9a5ceb590 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9edafda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e9a5d2d420 con 0x55e9a5a022f0 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ec2ffd640 1 -- 172.21.15.112:0/2854353597 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f9ebc042030 con 0x7f9ecc10da70 2024-04-25T16:47:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5dcaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5dcaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42038/0 (socket says 172.21.15.112:42038) 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 -- 172.21.15.112:0/984145440 learned_addr learned my addr 172.21.15.112:0/984145440 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5dcb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dcb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 -- 172.21.15.112:0/984145440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 msgr2=0x55e9a5dcb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5dcb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 -- 172.21.15.112:0/984145440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 msgr2=0x55e9a5dcb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dcb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 -- 172.21.15.112:0/984145440 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a5cf80d0 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.341+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5dcaaf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009f50 tx=0x7f9ec40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8743a46cb98899 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.342+0000 7f9eb7fff640 1 -- 172.21.15.112:0/984145440 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4004d20 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.342+0000 7f9eb7fff640 1 -- 172.21.15.112:0/984145440 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ec4004050 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.342+0000 7f9eb7fff640 1 -- 172.21.15.112:0/984145440 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005280 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.342+0000 7f9ec2ffd640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9eb80afc50 0x7f9eb80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.342+0000 7f9ec2ffd640 1 -- 172.21.15.112:0/2854353597 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8556+0+0 (secure 0 0 0) 0x7f9ebc0ff3b0 con 0x7f9ecc10da70 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.342+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9eb80afc50 0x7f9eb80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 -- 172.21.15.112:0/984145440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 msgr2=0x55e9a5dcaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9eb80afc50 0x7f9eb80b2160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55e9a5dcbea0 tx=0x7f9ec8005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5dcaaf0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009f50 tx=0x7f9ec40118d0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 -- 172.21.15.112:0/984145440 shutdown_connections 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5dcb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dcb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 --2- 172.21.15.112:0/984145440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5dcaaf0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 -- 172.21.15.112:0/984145440 >> 172.21.15.112:0/984145440 conn(0x55e9a5a1f300 msgr2=0x55e9a5dc72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 -- 172.21.15.112:0/984145440 shutdown_connections 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 -- 172.21.15.112:0/984145440 wait complete. 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 -- start start 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.343+0000 7f9edafda740 1 Processor -- start 2024-04-25T16:47:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5dd51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dd56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5ddb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e9a5dd7c20 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e9a5cb8b60 con 0x55e9a5a022f0 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e9a5dd7da0 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9edafda740 10 client.0 _ll_register_callbacks cb 0x55e9a5cdd2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dd56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed1eca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5dd51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dd56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42046/0 (socket says 172.21.15.112:42046) 2024-04-25T16:47:01.348 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed26cb640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5ddb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 -- 172.21.15.112:0/1646609073 learned_addr learned my addr 172.21.15.112:0/1646609073 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 -- 172.21.15.112:0/1646609073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a2b1b0 msgr2=0x55e9a5ddb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5ddb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 -- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 msgr2=0x55e9a5dd51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5dd51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 -- 172.21.15.112:0/1646609073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a5a43210 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5ddb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5dd51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dd56e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0fcb40 tx=0x7f9ebc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56e578b28b4da98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.344+0000 7f9eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9eb57fa640 1 -- 172.21.15.112:0/1646609073 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc0c6ca0 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9eb57fa640 1 -- 172.21.15.112:0/1646609073 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8556+0+0 (secure 0 0 0) 0x7f9ebc0fcd60 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec4004050 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.349 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9eb57fa640 1 -- 172.21.15.112:0/1646609073 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ebc0c7040 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 10 client.16101 fetch_fsmap learned FSMap version 29 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9a5b0f620 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9eb57fa640 1 -- 172.21.15.112:0/1646609073 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ec4004050 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9eb57fa640 1 -- 172.21.15.112:0/1646609073 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f9ebc0c37c0 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 20 client.16101 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a59e9d80 0x55e9a5de7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 4 client.16101 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9a5d3f070 con 0x55e9a59e9d80 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.345+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a59e9d80 0x55e9a5de7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.346+0000 7f9eb57fa640 10 client.16101 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.346+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a59e9d80 0x55e9a5de7720 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53a60f47a7d35d91 server_cookie=e829847c5acf801e in_seq=0 out_seq=0 2024-04-25T16:47:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9eb57fa640 1 -- 172.21.15.112:0/1646609073 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9a5d3f070 con 0x55e9a59e9d80 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9eb57fa640 10 client.16101 handle_command_reply: tid=0 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 client.16101 shutdown 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 msgr2=0x55e9a5dd56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dd56e0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0fcb40 tx=0x7f9ebc009650 comp rx=0 tx=0).stop 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 shutdown_connections 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a59e9d80 0x55e9a5de7720 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a2b1b0 0x55e9a5ddb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9eb57fa640 0 client.16101 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dd56e0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 --2- 172.21.15.112:0/1646609073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5dd51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 >> 172.21.15.112:0/1646609073 conn(0x55e9a5a1f300 msgr2=0x55e9a5dca920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 shutdown_connections 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 1 -- 172.21.15.112:0/1646609073 wait complete. 2024-04-25T16:47:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.347+0000 7f9edafda740 20 client.16101 trim_cache size 0 max 16384 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 Processor -- start 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 -- start start 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 0x55e9a5decef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5ded430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1b4630 con 0x55e9a59e9d80 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1b1f00 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.364+0000 7f9edafda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1afd50 con 0x55e9a5a022f0 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 0x55e9a5decef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed16c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5ded430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 0x55e9a5decef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42062/0 (socket says 172.21.15.112:42062) 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed16c9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5ded430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:37076/0 (socket says 172.21.15.112:37076) 2024-04-25T16:47:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 -- 172.21.15.112:0/2862557246 learned_addr learned my addr 172.21.15.112:0/2862557246 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 -- 172.21.15.112:0/2862557246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 msgr2=0x55e9a5ded430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5ded430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 -- 172.21.15.112:0/2862557246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 msgr2=0x55e9a5dce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 -- 172.21.15.112:0/2862557246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9a5dcbed0 con 0x55e9a59e9d80 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5ded430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 0x55e9a5decef0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55e9a5dc85e0 tx=0x7f9ec8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aca568308c3d110 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9eb77fe640 1 -- 172.21.15.112:0/2862557246 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8003c40 con 0x55e9a59e9d80 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9eb77fe640 1 -- 172.21.15.112:0/2862557246 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ec8032070 con 0x55e9a59e9d80 2024-04-25T16:47:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.365+0000 7f9eb77fe640 1 -- 172.21.15.112:0/2862557246 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8021790 con 0x55e9a59e9d80 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 -- 172.21.15.112:0/2862557246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 msgr2=0x55e9a5decef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 0x55e9a5decef0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x55e9a5dc85e0 tx=0x7f9ec8017000 comp rx=0 tx=0).stop 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 -- 172.21.15.112:0/2862557246 shutdown_connections 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a5a022f0 0x55e9a5ded430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 --2- 172.21.15.112:0/2862557246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a59e9d80 0x55e9a5decef0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 -- 172.21.15.112:0/2862557246 >> 172.21.15.112:0/2862557246 conn(0x55e9a5a1f300 msgr2=0x55e9a5dd3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 -- 172.21.15.112:0/2862557246 shutdown_connections 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.366+0000 7f9edafda740 1 -- 172.21.15.112:0/2862557246 wait complete. 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 -- start start 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 Processor -- start 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a59e9d80 0x55e9a5de13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5de1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dde8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1b03c0 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1b1830 con 0x55e9a5a022f0 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1b0a90 con 0x55e9a59e9d80 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9edafda740 10 client.0 _ll_register_callbacks cb 0x7f9ebc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9ed26cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dde8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9ed26cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dde8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42068/0 (socket says 172.21.15.112:42068) 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9ed1eca640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a59e9d80 0x55e9a5de13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 -- 172.21.15.112:0/3981216606 learned_addr learned my addr 172.21.15.112:0/3981216606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.368+0000 7f9ed16c9640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5de1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 -- 172.21.15.112:0/3981216606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a59e9d80 msgr2=0x55e9a5de13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a59e9d80 0x55e9a5de13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 -- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 msgr2=0x55e9a5de1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5de1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 -- 172.21.15.112:0/3981216606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9a5ddeef0 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a59e9d80 0x55e9a5de13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed26cb640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dde8e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009f50 tx=0x7f9ec4004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcab742cc609438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 1 -- 172.21.15.112:0/3981216606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4004460 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec8032070 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 1 -- 172.21.15.112:0/3981216606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8556+0+0 (secure 0 0 0) 0x7f9ec402e480 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 1 -- 172.21.15.112:0/3981216606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9ec402e680 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 10 client.16107 fetch_fsmap learned FSMap version 29 2024-04-25T16:47:01.372 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e9a5e4f3d0 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 1 -- 172.21.15.112:0/3981216606 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ec8032070 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 1 -- 172.21.15.112:0/3981216606 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f9ec400f7b0 con 0x55e9a5a0a4d0 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 10 client.16107 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 20 client.16107 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a5a2b1b0 0x55e9a5e519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 4 client.16107 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e9a5d38ee0 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a5a2b1b0 0x55e9a5e519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.369+0000 7f9eb57fa640 10 client.16107 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:01.373 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.371+0000 7f9ed1eca640 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a5a2b1b0 0x55e9a5e519a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c51d947d1e46ee server_cookie=e8371bfdd344901b in_seq=0 out_seq=0 2024-04-25T16:47:01.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:01 smithi133 ceph-mon[32691]: pgmap v267: 97 pgs: 97 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 385 op/s 2024-04-25T16:47:01.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:01 smithi116 ceph-mon[32712]: pgmap v267: 97 pgs: 97 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 385 op/s 2024-04-25T16:47:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.533+0000 7f9eb57fa640 1 -- 172.21.15.112:0/3981216606 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e9a5d38ee0 con 0x55e9a5a2b1b0 2024-04-25T16:47:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.533+0000 7f9eb57fa640 10 client.16107 handle_command_reply: tid=0 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 client.16107 shutdown 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 msgr2=0x55e9a5dde8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dde8e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009f50 tx=0x7f9ec4004c30 comp rx=0 tx=0).stop 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 shutdown_connections 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e9a5a0a4d0 0x55e9a5dde8e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e9a5a022f0 0x55e9a5de1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e9a5a2b1b0 0x55e9a5e519a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9eb57fa640 0 client.16107 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 --2- 172.21.15.112:0/3981216606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e9a59e9d80 0x55e9a5de13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 >> 172.21.15.112:0/3981216606 conn(0x55e9a5a1f300 msgr2=0x55e9a5dce740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 shutdown_connections 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 1 -- 172.21.15.112:0/3981216606 wait complete. 2024-04-25T16:47:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.534+0000 7f9edafda740 20 client.16107 trim_cache size 0 max 16384 2024-04-25T16:47:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:47:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:47:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:47:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9eb80afc50 msgr2=0x7f9eb80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9eb80afc50 0x7f9eb80b2160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55e9a5dcbea0 tx=0x7f9ec8005f90 comp rx=0 tx=0).stop 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 msgr2=0x7f9ecc1a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc1a2af0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005ab0 tx=0x7f9ebc031a20 comp rx=0 tx=0).stop 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 shutdown_connections 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9eb80afc50 0x7f9eb80b2160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ecc1103d0 0x7f9ecc19c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ecc10da70 0x7f9ecc1a2af0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 --2- 172.21.15.112:0/2854353597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ecc0763f0 0x7f9ecc1a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 >> 172.21.15.112:0/2854353597 conn(0x7f9ecc0ff1e0 msgr2=0x7f9ecc101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 shutdown_connections 2024-04-25T16:47:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:01.535+0000 7f9ed412e640 1 -- 172.21.15.112:0/2854353597 wait complete. 2024-04-25T16:47:01.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:01 smithi112 ceph-mon[29122]: pgmap v267: 97 pgs: 97 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 385 op/s 2024-04-25T16:47:01.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 915916 10.90 MB/sec execute 307 sec latency 58.529 ms 2024-04-25T16:47:02.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 917037 10.91 MB/sec execute 308 sec latency 65.311 ms 2024-04-25T16:47:03.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:03 smithi133 ceph-mon[32691]: pgmap v268: 97 pgs: 97 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-25T16:47:03.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:03 smithi116 ceph-mon[32712]: pgmap v268: 97 pgs: 97 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-25T16:47:03.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:03 smithi112 ceph-mon[29122]: pgmap v268: 97 pgs: 97 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.9 MiB/s wr, 196 op/s 2024-04-25T16:47:03.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 918826 10.92 MB/sec execute 309 sec latency 47.460 ms 2024-04-25T16:47:04.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 920835 10.93 MB/sec execute 310 sec latency 98.491 ms 2024-04-25T16:47:05.397 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:05 smithi112 ceph-mon[29122]: pgmap v269: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 392 op/s 2024-04-25T16:47:05.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:05 smithi133 ceph-mon[32691]: pgmap v269: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 392 op/s 2024-04-25T16:47:05.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:05 smithi116 ceph-mon[32712]: pgmap v269: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.4 MiB/s wr, 392 op/s 2024-04-25T16:47:05.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 922601 10.91 MB/sec execute 311 sec latency 46.646 ms 2024-04-25T16:47:06.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 923992 10.92 MB/sec execute 312 sec latency 26.462 ms 2024-04-25T16:47:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:07 smithi133 ceph-mon[32691]: pgmap v270: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 300 op/s 2024-04-25T16:47:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:07 smithi133 ceph-mon[32691]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T16:47:07.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:07 smithi116 ceph-mon[32712]: pgmap v270: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 300 op/s 2024-04-25T16:47:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:07 smithi116 ceph-mon[32712]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T16:47:07.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:07 smithi112 ceph-mon[29122]: pgmap v270: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 300 op/s 2024-04-25T16:47:07.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:07 smithi112 ceph-mon[29122]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T16:47:07.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 924372 10.88 MB/sec execute 313 sec latency 27.246 ms 2024-04-25T16:47:08.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 925747 10.86 MB/sec execute 314 sec latency 29.179 ms 2024-04-25T16:47:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:09 smithi133 ceph-mon[32691]: pgmap v272: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-04-25T16:47:09.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:09 smithi116 ceph-mon[32712]: pgmap v272: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-04-25T16:47:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:09 smithi112 ceph-mon[29122]: pgmap v272: 97 pgs: 97 active+clean; 69 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-04-25T16:47:09.886 INFO:tasks.workunit.client.0.smithi112.stdout: 1 927895 10.87 MB/sec execute 315 sec latency 15.016 ms 2024-04-25T16:47:10.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 929709 10.86 MB/sec execute 316 sec latency 112.724 ms 2024-04-25T16:47:11.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:11 smithi133 ceph-mon[32691]: pgmap v273: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-04-25T16:47:11.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:11 smithi116 ceph-mon[32712]: pgmap v273: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-04-25T16:47:11.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:11 smithi112 ceph-mon[29122]: pgmap v273: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-04-25T16:47:11.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 931616 10.87 MB/sec execute 317 sec latency 13.770 ms 2024-04-25T16:47:12.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 934077 10.87 MB/sec execute 318 sec latency 17.712 ms 2024-04-25T16:47:13.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:13 smithi112 ceph-mon[29122]: pgmap v274: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-04-25T16:47:13.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:13 smithi112 ceph-mon[29122]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T16:47:13.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:13 smithi133 ceph-mon[32691]: pgmap v274: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-04-25T16:47:13.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:13 smithi133 ceph-mon[32691]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T16:47:13.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:13 smithi116 ceph-mon[32712]: pgmap v274: 97 pgs: 97 active+clean; 73 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-04-25T16:47:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:13 smithi116 ceph-mon[32712]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T16:47:13.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 935867 10.87 MB/sec execute 319 sec latency 17.084 ms 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 -- 172.21.15.112:0/4031726778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 msgr2=0x7f8d38105070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/4031726778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d38105070 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009900 tx=0x7f8d3402a410 comp rx=0 tx=0).stop 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 -- 172.21.15.112:0/4031726778 shutdown_connections 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/4031726778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3810b6c0 0x7f8d3810dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/4031726778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810e0b0 0x7f8d381105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/4031726778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d38105070 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 -- 172.21.15.112:0/4031726778 >> 172.21.15.112:0/4031726778 conn(0x7f8d380fe260 msgr2=0x7f8d38100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 -- 172.21.15.112:0/4031726778 shutdown_connections 2024-04-25T16:47:14.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.639+0000 7f8d3fc42640 1 -- 172.21.15.112:0/4031726778 wait complete. 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 Processor -- start 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 -- start start 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d381a2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810b6c0 0x7f8d381a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3810e0b0 0x7f8d3819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38103750 con 0x7f8d38104c60 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381035d0 con 0x7f8d3810b6c0 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3fc42640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381038d0 con 0x7f8d3810e0b0 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d381a2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d381a2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54468/0 (socket says 172.21.15.112:54468) 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2202822881 learned_addr learned my addr 172.21.15.112:0/2202822881 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:14.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810b6c0 0x7f8d381a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3810e0b0 0x7f8d3819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2202822881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3810e0b0 msgr2=0x7f8d3819c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3810e0b0 0x7f8d3819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2202822881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810b6c0 msgr2=0x7f8d381a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810b6c0 0x7f8d381a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2202822881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3819cea0 con 0x7f8d38104c60 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810b6c0 0x7f8d381a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.640+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d381a2590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005940 tx=0x7f8d340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.641+0000 7f8d2effd640 1 -- 172.21.15.112:0/2202822881 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3403f030 con 0x7f8d38104c60 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.641+0000 7f8d2effd640 1 -- 172.21.15.112:0/2202822881 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340044c0 con 0x7f8d38104c60 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.641+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3819d130 con 0x7f8d38104c60 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.641+0000 7f8d2effd640 1 -- 172.21.15.112:0/2202822881 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34035070 con 0x7f8d38104c60 2024-04-25T16:47:14.642 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.641+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3819d620 con 0x7f8d38104c60 2024-04-25T16:47:14.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.643+0000 7f8d2effd640 1 -- 172.21.15.112:0/2202822881 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f8d34042030 con 0x7f8d38104c60 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 Processor -- start 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 -- start start 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f382fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f382fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f382fe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f38264460 con 0x563f37f3d4d0 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563f3821e590 con 0x563f37f352f0 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d46bce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563f38260420 con 0x563f37f5e1b0 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d2effd640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8d240afc50 0x7f8d240b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d3d1dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f382fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f382fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f382fe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:52918/0 (socket says 172.21.15.112:52918) 2024-04-25T16:47:14.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d3d9de640 1 -- 172.21.15.112:0/356325249 learned_addr learned my addr 172.21.15.112:0/356325249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8d240afc50 0x7f8d240b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f382fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.644+0000 7f8d2effd640 1 -- 172.21.15.112:0/2202822881 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f8d340ff4a0 con 0x7f8d38104c60 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3d9de640 1 -- 172.21.15.112:0/356325249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 msgr2=0x563f382fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f382fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3d9de640 1 -- 172.21.15.112:0/356325249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 msgr2=0x563f382fe570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f382fe570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3d9de640 1 -- 172.21.15.112:0/356325249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f3822b0d0 con 0x563f37f352f0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8d240afc50 0x7f8d240b2160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x563f382fac60 tx=0x7f8d300091e0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f382fe030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8d34006010 tx=0x7f8d3403b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e66aeb1b51841f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d23fff640 1 -- 172.21.15.112:0/356325249 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340b00d0 con 0x563f37f352f0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d23fff640 1 -- 172.21.15.112:0/356325249 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340c7cf0 con 0x563f37f352f0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.645+0000 7f8d23fff640 1 -- 172.21.15.112:0/356325249 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340c7850 con 0x563f37f352f0 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 -- 172.21.15.112:0/356325249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 msgr2=0x563f382fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f382fe030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8d34006010 tx=0x7f8d3403b4f0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.651 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 -- 172.21.15.112:0/356325249 shutdown_connections 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f382fe570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f382fe030 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 --2- 172.21.15.112:0/356325249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f382fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 -- 172.21.15.112:0/356325249 >> 172.21.15.112:0/356325249 conn(0x563f37f52300 msgr2=0x563f382fa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 -- 172.21.15.112:0/356325249 shutdown_connections 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 -- 172.21.15.112:0/356325249 wait complete. 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.646+0000 7f8d46bce740 1 -- start start 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 Processor -- start 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f383081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38308710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f3830ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x563f3830a9e0 con 0x563f37f3d4d0 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563f381ebb60 con 0x563f37f352f0 2024-04-25T16:47:14.652 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563f3830ab60 con 0x563f37f5e1b0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d46bce740 10 client.0 _ll_register_callbacks cb 0x563f382102b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d1dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38308710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f383081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f383081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:52920/0 (socket says 172.21.15.112:52920) 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2300367282 learned_addr learned my addr 172.21.15.112:0/2300367282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f3830ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2300367282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 msgr2=0x563f3830ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f3830ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2300367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 msgr2=0x563f38308710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38308710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d9de640 1 -- 172.21.15.112:0/2300367282 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f37f76210 con 0x563f37f352f0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.647+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38308710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f383081b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d340d0610 tx=0x7f8d340fd050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f408cfb48cd93421 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:47:14.653 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d217fa640 1 -- 172.21.15.112:0/2300367282 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34049040 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d217fa640 1 -- 172.21.15.112:0/2300367282 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f8d340c7a50 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d340c7cf0 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d217fa640 1 -- 172.21.15.112:0/2300367282 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d340b00d0 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f38042620 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d217fa640 1 -- 172.21.15.112:0/2300367282 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d340c7cf0 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d217fa640 1 -- 172.21.15.112:0/2300367282 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8d340fe070 con 0x563f37f352f0 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 20 client.25817 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:14.654 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37f1cd80 0x563f3831a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 4 client.25817 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.648+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f38272070 con 0x563f37f1cd80 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.649+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37f1cd80 0x563f3831a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.649+0000 7f8d217fa640 10 client.25817 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.649+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37f1cd80 0x563f3831a280 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378187a2d3da5a5a server_cookie=73244fe8dd6512b4 in_seq=0 out_seq=0 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d217fa640 1 -- 172.21.15.112:0/2300367282 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f38272070 con 0x563f37f1cd80 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d217fa640 10 client.25817 handle_command_reply: tid=0 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 client.25817 shutdown 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 msgr2=0x563f383081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f383081b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8d340d0610 tx=0x7f8d340fd050 comp rx=0 tx=0).stop 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 shutdown_connections 2024-04-25T16:47:14.655 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37f1cd80 0x563f3831a280 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f3830ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38308710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d217fa640 0 client.25817 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 --2- 172.21.15.112:0/2300367282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f383081b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 >> 172.21.15.112:0/2300367282 conn(0x563f37f52300 msgr2=0x563f382fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 shutdown_connections 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 1 -- 172.21.15.112:0/2300367282 wait complete. 2024-04-25T16:47:14.656 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.650+0000 7f8d46bce740 20 client.25817 trim_cache size 0 max 16384 2024-04-25T16:47:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 Processor -- start 2024-04-25T16:47:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 -- start start 2024-04-25T16:47:14.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38301ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f3d4d0 0x563f3831f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 0x563f3831feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381b4660 con 0x563f37f5e1b0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381b1f30 con 0x563f37f352f0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d46bce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381afda0 con 0x563f37f3d4d0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.666+0000 7f8d3e1df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 0x563f3831feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3d9de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38301ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 0x563f3831feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54482/0 (socket says 172.21.15.112:54482) 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 -- 172.21.15.112:0/1656380369 learned_addr learned my addr 172.21.15.112:0/1656380369 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f3d4d0 0x563f3831f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 -- 172.21.15.112:0/1656380369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f3d4d0 msgr2=0x563f3831f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f3d4d0 0x563f3831f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 -- 172.21.15.112:0/1656380369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 msgr2=0x563f38301ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38301ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 -- 172.21.15.112:0/1656380369 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f382fe8a0 con 0x563f37f5e1b0 2024-04-25T16:47:14.671 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38301ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f3d4d0 0x563f3831f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 0x563f3831feb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8d30014200 tx=0x7f8d30009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e766151720509d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d237fe640 1 -- 172.21.15.112:0/1656380369 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004b20 con 0x563f37f5e1b0 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d237fe640 1 -- 172.21.15.112:0/1656380369 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d30004cc0 con 0x563f37f5e1b0 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.667+0000 7f8d237fe640 1 -- 172.21.15.112:0/1656380369 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004030 con 0x563f37f5e1b0 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 -- 172.21.15.112:0/1656380369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 msgr2=0x563f3831feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 0x563f3831feb0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8d30014200 tx=0x7f8d30009500 comp rx=0 tx=0).stop 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 -- 172.21.15.112:0/1656380369 shutdown_connections 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f3d4d0 0x563f3831f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f5e1b0 0x563f3831feb0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1656380369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38301ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 -- 172.21.15.112:0/1656380369 >> 172.21.15.112:0/1656380369 conn(0x563f37f52300 msgr2=0x563f383017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 -- 172.21.15.112:0/1656380369 shutdown_connections 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.668+0000 7f8d46bce740 1 -- 172.21.15.112:0/1656380369 wait complete. 2024-04-25T16:47:14.672 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 -- start start 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 Processor -- start 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38314710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38311ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f38312010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381b03f0 con 0x563f37f3d4d0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381b1860 con 0x563f37f352f0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d381b0ac0 con 0x563f37f5e1b0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d46bce740 10 client.0 _ll_register_callbacks cb 0x7f8d28009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d3d1dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38311ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d3d1dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38311ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:54494/0 (socket says 172.21.15.112:54494) 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d3d1dd640 1 -- 172.21.15.112:0/1842360277 learned_addr learned my addr 172.21.15.112:0/1842360277 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38314710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.669+0000 7f8d3e1df640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f38312010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d3d1dd640 1 -- 172.21.15.112:0/1842360277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 msgr2=0x563f38312010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f38312010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d3d1dd640 1 -- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 msgr2=0x563f38314710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38314710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d3d1dd640 1 -- 172.21.15.112:0/1842360277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f38312700 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d3d1dd640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38311ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x563f382f4690 tx=0x7f8d28012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b2eeb70be41090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d217fa640 1 -- 172.21.15.112:0/1842360277 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2800fc10 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d217fa640 1 -- 172.21.15.112:0/1842360277 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f8d2800fdb0 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d30004cc0 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d217fa640 1 -- 172.21.15.112:0/1842360277 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d28030050 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 10 client.16125 fetch_fsmap learned FSMap version 29 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563f383823b0 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d217fa640 1 -- 172.21.15.112:0/1842360277 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d30004cc0 con 0x563f37f3d4d0 2024-04-25T16:47:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d217fa640 1 -- 172.21.15.112:0/1842360277 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f8d280220f0 con 0x563f37f3d4d0 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 10 client.16125 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 20 client.16125 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.670+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37ef21c0 0x563f38384b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.671+0000 7f8d46bce740 4 client.16125 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.671+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f3826bee0 con 0x563f37ef21c0 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.671+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37ef21c0 0x563f38384b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.671+0000 7f8d217fa640 10 client.16125 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.671+0000 7f8d3d9de640 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37ef21c0 0x563f38384b30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc512113f3846d3 server_cookie=c4fd717fa25a67e8 in_seq=0 out_seq=0 2024-04-25T16:47:14.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 937978 10.88 MB/sec execute 320 sec latency 127.859 ms 2024-04-25T16:47:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d217fa640 1 -- 172.21.15.112:0/1842360277 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f3826bee0 con 0x563f37ef21c0 2024-04-25T16:47:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d217fa640 10 client.16125 handle_command_reply: tid=0 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 client.16125 shutdown 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 msgr2=0x563f38311ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38311ad0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x563f382f4690 tx=0x7f8d28012ed0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 shutdown_connections 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x563f37ef21c0 0x563f38384b30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563f37f5e1b0 0x563f38312010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x563f37f3d4d0 0x563f38311ad0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 --2- 172.21.15.112:0/1842360277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563f37f352f0 0x563f38314710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 >> 172.21.15.112:0/1842360277 conn(0x563f37f52300 msgr2=0x563f382fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d217fa640 0 client.16125 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 shutdown_connections 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.891+0000 7f8d46bce740 1 -- 172.21.15.112:0/1842360277 wait complete. 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.892+0000 7f8d46bce740 20 client.16125 trim_cache size 0 max 16384 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:47:14.891 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.892+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8d240afc50 msgr2=0x7f8d240b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.892+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8d240afc50 0x7f8d240b2160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x563f382fac60 tx=0x7f8d300091e0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.892+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 msgr2=0x7f8d381a2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.892+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d381a2590 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005940 tx=0x7f8d340040c0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 shutdown_connections 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8d240afc50 0x7f8d240b2160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.892 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d3810e0b0 0x7f8d3819c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d3810b6c0 0x7f8d381a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 --2- 172.21.15.112:0/2202822881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8d38104c60 0x7f8d381a2590 secure :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d34005940 tx=0x7f8d340040c0 comp rx=0 tx=0).stop 2024-04-25T16:47:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 >> 172.21.15.112:0/2202822881 conn(0x7f8d380fe260 msgr2=0x7f8d3810ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 shutdown_connections 2024-04-25T16:47:14.893 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:14.893+0000 7f8d3fc42640 1 -- 172.21.15.112:0/2202822881 wait complete. 2024-04-25T16:47:15.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:15 smithi112 ceph-mon[29122]: pgmap v276: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-04-25T16:47:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:15 smithi133 ceph-mon[32691]: pgmap v276: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-04-25T16:47:15.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:15 smithi116 ceph-mon[32712]: pgmap v276: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-04-25T16:47:15.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 938896 10.85 MB/sec execute 321 sec latency 143.338 ms 2024-04-25T16:47:16.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 941350 10.86 MB/sec execute 322 sec latency 20.920 ms 2024-04-25T16:47:17.345 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:17 smithi112 ceph-mon[29122]: pgmap v277: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2024-04-25T16:47:17.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:17 smithi133 ceph-mon[32691]: pgmap v277: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2024-04-25T16:47:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:17 smithi116 ceph-mon[32712]: pgmap v277: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 200 op/s 2024-04-25T16:47:17.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 942987 10.86 MB/sec execute 323 sec latency 16.759 ms 2024-04-25T16:47:18.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 945244 10.86 MB/sec execute 324 sec latency 18.953 ms 2024-04-25T16:47:19.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:19 smithi112 ceph-mon[29122]: pgmap v278: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-04-25T16:47:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:19 smithi133 ceph-mon[32691]: pgmap v278: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-04-25T16:47:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:19 smithi116 ceph-mon[32712]: pgmap v278: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-04-25T16:47:19.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 946881 10.85 MB/sec execute 325 sec latency 15.548 ms 2024-04-25T16:47:20.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 948862 10.86 MB/sec execute 326 sec latency 113.254 ms 2024-04-25T16:47:21.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:21 smithi112 ceph-mon[29122]: pgmap v279: 97 pgs: 97 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:47:21.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:21 smithi133 ceph-mon[32691]: pgmap v279: 97 pgs: 97 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:47:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:21 smithi116 ceph-mon[32712]: pgmap v279: 97 pgs: 97 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:47:21.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 950544 10.84 MB/sec execute 327 sec latency 26.950 ms 2024-04-25T16:47:22.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 952730 10.85 MB/sec execute 328 sec latency 15.668 ms 2024-04-25T16:47:23.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:23 smithi133 ceph-mon[32691]: pgmap v280: 97 pgs: 97 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:47:23.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:23 smithi116 ceph-mon[32712]: pgmap v280: 97 pgs: 97 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:47:23.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:23 smithi112 ceph-mon[29122]: pgmap v280: 97 pgs: 97 active+clean; 74 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-04-25T16:47:23.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 955094 10.85 MB/sec execute 329 sec latency 17.336 ms 2024-04-25T16:47:24.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 956687 10.85 MB/sec execute 330 sec latency 20.082 ms 2024-04-25T16:47:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:25 smithi133 ceph-mon[32691]: pgmap v281: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 399 op/s 2024-04-25T16:47:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:25 smithi116 ceph-mon[32712]: pgmap v281: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 399 op/s 2024-04-25T16:47:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:25 smithi112 ceph-mon[29122]: pgmap v281: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 399 op/s 2024-04-25T16:47:25.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 959082 10.85 MB/sec execute 331 sec latency 92.532 ms 2024-04-25T16:47:26.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 960547 10.84 MB/sec execute 332 sec latency 21.524 ms 2024-04-25T16:47:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:27 smithi133 ceph-mon[32691]: pgmap v282: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-04-25T16:47:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:27 smithi116 ceph-mon[32712]: pgmap v282: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-04-25T16:47:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:27 smithi112 ceph-mon[29122]: pgmap v282: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-04-25T16:47:27.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 962996 10.86 MB/sec execute 333 sec latency 18.073 ms 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 -- 172.21.15.112:0/2535804178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 msgr2=0x7f53541105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 --2- 172.21.15.112:0/2535804178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f53541105e0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f534c009ca0 tx=0x7f534c02a9e0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 -- 172.21.15.112:0/2535804178 shutdown_connections 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 --2- 172.21.15.112:0/2535804178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f53541105e0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 --2- 172.21.15.112:0/2535804178 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f535410b740 0x7f535410db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 --2- 172.21.15.112:0/2535804178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5354104ce0 0x7f53541050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 -- 172.21.15.112:0/2535804178 >> 172.21.15.112:0/2535804178 conn(0x7f5354079170 msgr2=0x7f53540795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 -- 172.21.15.112:0/2535804178 shutdown_connections 2024-04-25T16:47:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.995+0000 7f5358fa8640 1 -- 172.21.15.112:0/2535804178 wait complete. 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 Processor -- start 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 -- start start 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5354104ce0 0x7f535419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535410b740 0x7f535419acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f535419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5354113d50 con 0x7f535410e0f0 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5354113bd0 con 0x7f5354104ce0 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.996+0000 7f5358fa8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5354113ed0 con 0x7f535410b740 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f535419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f535259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5354104ce0 0x7f535419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f535419efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32780/0 (socket says 172.21.15.112:32780) 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 -- 172.21.15.112:0/1303783622 learned_addr learned my addr 172.21.15.112:0/1303783622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5351d9b640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535410b740 0x7f535419acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 -- 172.21.15.112:0/1303783622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535410b740 msgr2=0x7f535419acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535410b740 0x7f535419acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 -- 172.21.15.112:0/1303783622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5354104ce0 msgr2=0x7f535419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5354104ce0 0x7f535419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 -- 172.21.15.112:0/1303783622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535419f5f0 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f535259c640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5354104ce0 0x7f535419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5352d9d640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f535419efb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f534c006010 tx=0x7f534c02fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f53437fe640 1 -- 172.21.15.112:0/1303783622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c033040 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535419f880 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f53437fe640 1 -- 172.21.15.112:0/1303783622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534c02e7a0 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f53437fe640 1 -- 172.21.15.112:0/1303783622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c043720 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:27.997+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535419fd10 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.000+0000 7f535fe51740 1 Processor -- start 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.000+0000 7f53437fe640 1 -- 172.21.15.112:0/1303783622 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f534c036070 con 0x7f535410e0f0 2024-04-25T16:47:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.000+0000 7f535fe51740 1 -- start start 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535fe51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c81af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535fe51740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c82030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535fe51740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48c14d0 0x5592d4c82570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535fe51740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592d4be8460 con 0x5592d48c14d0 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535fe51740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5592d4ba2590 con 0x5592d48b92f0 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535fe51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592d4be4420 con 0x5592d48e21b0 2024-04-25T16:47:28.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c82030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c81af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c82030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:43010/0 (socket says 172.21.15.112:43010) 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c81af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:41114/0 (socket says 172.21.15.112:41114) 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 -- 172.21.15.112:0/3419488198 learned_addr learned my addr 172.21.15.112:0/3419488198 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f5351d9b640 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48c14d0 0x5592d4c82570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f53437fe640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f533c0afc50 0x7f533c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 -- 172.21.15.112:0/3419488198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 msgr2=0x5592d4c81af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c81af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 -- 172.21.15.112:0/3419488198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48c14d0 msgr2=0x5592d4c82570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48c14d0 0x5592d4c82570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 -- 172.21.15.112:0/3419488198 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d4baf0d0 con 0x5592d48b92f0 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f5352d9d640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f533c0afc50 0x7f533c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f5351d9b640 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c81af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f535259c640 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c82030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53440104d0 tx=0x7f5344010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1cc4230cb15ae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f53437fe640 1 -- 172.21.15.112:0/1303783622 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f534c0ffdb0 con 0x7f535410e0f0 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.001+0000 7f5340ff9640 1 -- 172.21.15.112:0/3419488198 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344011760 con 0x5592d48b92f0 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f5340ff9640 1 -- 172.21.15.112:0/3419488198 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344011db0 con 0x5592d48b92f0 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f5340ff9640 1 -- 172.21.15.112:0/3419488198 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534401cd90 con 0x5592d48b92f0 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 -- 172.21.15.112:0/3419488198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 msgr2=0x5592d4c82030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c82030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f53440104d0 tx=0x7f5344010990 comp rx=0 tx=0).stop 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f5352d9d640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f533c0afc50 0x7f533c0b2160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f534c0c7f40 tx=0x7f534c08e920 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 -- 172.21.15.112:0/3419488198 shutdown_connections 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48c14d0 0x5592d4c82570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c82030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 --2- 172.21.15.112:0/3419488198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c81af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 -- 172.21.15.112:0/3419488198 >> 172.21.15.112:0/3419488198 conn(0x5592d48d6300 msgr2=0x5592d4c7e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.002+0000 7f535fe51740 1 -- 172.21.15.112:0/3419488198 shutdown_connections 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 -- 172.21.15.112:0/3419488198 wait complete. 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 -- start start 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 Processor -- start 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 0x5592d4c8c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4c8c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c92180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5592d4c8ec20 con 0x5592d48b92f0 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5592d4b6fb60 con 0x5592d48c14d0 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5592d4c8eda0 con 0x5592d48e21b0 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535fe51740 10 client.0 _ll_register_callbacks cb 0x5592d4b942b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 0x5592d4c8c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 0x5592d4c8c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32790/0 (socket says 172.21.15.112:32790) 2024-04-25T16:47:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f5351d9b640 1 -- 172.21.15.112:0/3322288195 learned_addr learned my addr 172.21.15.112:0/3322288195 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f535259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4c8c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.003+0000 7f5351d9b640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c92180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5351d9b640 1 -- 172.21.15.112:0/3322288195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 msgr2=0x5592d4c92180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5351d9b640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c92180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5351d9b640 1 -- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 msgr2=0x5592d4c8c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5351d9b640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4c8c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5351d9b640 1 -- 172.21.15.112:0/3322288195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d48fa210 con 0x5592d48b92f0 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f535259c640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4c8c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5351d9b640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 0x5592d4c8c1e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5348009200 tx=0x7f5348004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d669407bd6b7d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5339ffb640 1 -- 172.21.15.112:0/3322288195 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534800fb90 con 0x5592d48b92f0 2024-04-25T16:47:28.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5339ffb640 1 -- 172.21.15.112:0/3322288195 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f534800fd30 con 0x5592d48b92f0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5344011db0 con 0x5592d48b92f0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5339ffb640 1 -- 172.21.15.112:0/3322288195 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5348005020 con 0x5592d48b92f0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f535fe51740 10 client.16137 fetch_fsmap learned FSMap version 29 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592d49c6620 con 0x5592d48b92f0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5339ffb640 1 -- 172.21.15.112:0/3322288195 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5344011db0 con 0x5592d48b92f0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f5339ffb640 1 -- 172.21.15.112:0/3322288195 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5348004cd0 con 0x5592d48b92f0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f535fe51740 10 client.16137 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.004+0000 7f535fe51740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f535fe51740 20 client.16137 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f535fe51740 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48a0d80 0x5592d4c9e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f535fe51740 4 client.16137 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592d4bf6070 con 0x5592d48a0d80 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f535259c640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48a0d80 0x5592d4c9e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f535259c640 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48a0d80 0x5592d4c9e7b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a85de870b93c1a8 server_cookie=adaa80ca7333995d in_seq=0 out_seq=0 2024-04-25T16:47:28.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.005+0000 7f5339ffb640 10 client.16137 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f5339ffb640 1 -- 172.21.15.112:0/3322288195 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592d4bf6070 con 0x5592d48a0d80 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f5339ffb640 10 client.16137 handle_command_reply: tid=0 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 client.16137 shutdown 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 msgr2=0x5592d4c8c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 0x5592d4c8c1e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5348009200 tx=0x7f5348004730 comp rx=0 tx=0).stop 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 shutdown_connections 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48a0d80 0x5592d4c9e7b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48e21b0 0x5592d4c92180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f5339ffb640 0 client.16137 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4c8c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 --2- 172.21.15.112:0/3322288195 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48b92f0 0x5592d4c8c1e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 >> 172.21.15.112:0/3322288195 conn(0x5592d48d6300 msgr2=0x5592d4c81920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 shutdown_connections 2024-04-25T16:47:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 1 -- 172.21.15.112:0/3322288195 wait complete. 2024-04-25T16:47:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.007+0000 7f535fe51740 20 client.16137 trim_cache size 0 max 16384 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 Processor -- start 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 -- start start 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48b92f0 0x5592d4c85ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4ca3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4ca44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53541b2470 con 0x5592d48e21b0 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f53541afd40 con 0x5592d48c14d0 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535fe51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53541adbb0 con 0x5592d48b92f0 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4ca44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4ca3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4ca44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32794/0 (socket says 172.21.15.112:32794) 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f5351d9b640 1 -- 172.21.15.112:0/1591101585 learned_addr learned my addr 172.21.15.112:0/1591101585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f535259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4ca3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:43018/0 (socket says 172.21.15.112:43018) 2024-04-25T16:47:28.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.023+0000 7f5351d9b640 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48b92f0 0x5592d4c85ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f5351d9b640 1 -- 172.21.15.112:0/1591101585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48b92f0 msgr2=0x5592d4c85ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f5351d9b640 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48b92f0 0x5592d4c85ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f5351d9b640 1 -- 172.21.15.112:0/1591101585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 msgr2=0x5592d4ca3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f5351d9b640 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4ca3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f5351d9b640 1 -- 172.21.15.112:0/1591101585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592d4c8cec0 con 0x5592d48e21b0 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f535259c640 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4ca3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f5351d9b640 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4ca44f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x5592d4c9b4a0 tx=0x7f5348009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e766da8085a641d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f533bfff640 1 -- 172.21.15.112:0/1591101585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534800fb90 con 0x5592d48e21b0 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f533bfff640 1 -- 172.21.15.112:0/1591101585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534800fd30 con 0x5592d48e21b0 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.024+0000 7f533bfff640 1 -- 172.21.15.112:0/1591101585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53480047a0 con 0x5592d48e21b0 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 -- 172.21.15.112:0/1591101585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 msgr2=0x5592d4ca44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4ca44f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x5592d4c9b4a0 tx=0x7f5348009a20 comp rx=0 tx=0).stop 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 -- 172.21.15.112:0/1591101585 shutdown_connections 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4ca44f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48c14d0 0x5592d4ca3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 --2- 172.21.15.112:0/1591101585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48b92f0 0x5592d4c85ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 -- 172.21.15.112:0/1591101585 >> 172.21.15.112:0/1591101585 conn(0x5592d48d6300 msgr2=0x5592d4c857b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 -- 172.21.15.112:0/1591101585 shutdown_connections 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 -- 172.21.15.112:0/1591101585 wait complete. 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 -- start start 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.025+0000 7f535fe51740 1 Processor -- start 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c98a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 0x5592d4cff830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4cffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f53541ae200 con 0x5592d48e21b0 2024-04-25T16:47:28.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f53541af670 con 0x5592d48b92f0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f53541ae8d0 con 0x5592d48c14d0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535fe51740 10 client.0 _ll_register_callbacks cb 0x7f53480116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4cffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 0x5592d4cff830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 0x5592d4cff830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:41132/0 (socket says 172.21.15.112:41132) 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f5351d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4cffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32804/0 (socket says 172.21.15.112:32804) 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 -- 172.21.15.112:0/2722456080 learned_addr learned my addr 172.21.15.112:0/2722456080 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f5351d9b640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 -- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 msgr2=0x5592d4c98a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c98a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 -- 172.21.15.112:0/2722456080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 msgr2=0x5592d4cffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4cffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 -- 172.21.15.112:0/2722456080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592d4d003e0 con 0x5592d48c14d0 2024-04-25T16:47:28.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f5351d9b640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4cffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f535259c640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 0x5592d4cff830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53440104d0 tx=0x7f534401c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a49ea64c44f7ac04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.026+0000 7f5339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f5339ffb640 1 -- 172.21.15.112:0/2722456080 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534402e070 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f5339ffb640 1 -- 172.21.15.112:0/2722456080 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8600+0+0 (secure 0 0 0) 0x7f534401c940 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534800fd30 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f5339ffb640 1 -- 172.21.15.112:0/2722456080 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f534403a050 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 10 client.25765 fetch_fsmap learned FSMap version 29 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5592d4d0af70 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f5339ffb640 1 -- 172.21.15.112:0/2722456080 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f534800fd30 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f5339ffb640 1 -- 172.21.15.112:0/2722456080 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5344003680 con 0x5592d48c14d0 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 10 client.25765 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 20 client.25765 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:28.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48761c0 0x5592d4d0d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 4 client.25765 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592d4befee0 con 0x5592d48761c0 2024-04-25T16:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.027+0000 7f5351d9b640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48761c0 0x5592d4d0d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.028+0000 7f5339ffb640 10 client.25765 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:28.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.028+0000 7f5351d9b640 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48761c0 0x5592d4d0d310 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb92fd5450285cb server_cookie=ae881eda1b263fe in_seq=0 out_seq=0 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f5339ffb640 1 -- 172.21.15.112:0/2722456080 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592d4befee0 con 0x5592d48761c0 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f5339ffb640 10 client.25765 handle_command_reply: tid=0 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 client.25765 shutdown 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 msgr2=0x5592d4cff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 0x5592d4cff830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53440104d0 tx=0x7f534401c6b0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 shutdown_connections 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5592d48761c0 0x5592d4d0d310 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592d48c14d0 0x5592d4cff830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f5339ffb640 0 client.25765 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:28.224 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5592d48e21b0 0x5592d4cffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 --2- 172.21.15.112:0/2722456080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592d48b92f0 0x5592d4c98a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 >> 172.21.15.112:0/2722456080 conn(0x5592d48d6300 msgr2=0x5592d4c81880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 shutdown_connections 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.225+0000 7f535fe51740 1 -- 172.21.15.112:0/2722456080 wait complete. 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f535fe51740 20 client.25765 trim_cache size 0 max 16384 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:47:28.225 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:47:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f533c0afc50 msgr2=0x7f533c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f533c0afc50 0x7f533c0b2160 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f534c0c7f40 tx=0x7f534c08e920 comp rx=0 tx=0).stop 2024-04-25T16:47:28.226 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 msgr2=0x7f535419efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f535419efb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f534c006010 tx=0x7f534c02fbe0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 shutdown_connections 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f533c0afc50 0x7f533c0b2160 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f535410b740 0x7f535419acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f535410e0f0 0x7f535419efb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 --2- 172.21.15.112:0/1303783622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5354104ce0 0x7f535419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 >> 172.21.15.112:0/1303783622 conn(0x7f5354079170 msgr2=0x7f5354101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 shutdown_connections 2024-04-25T16:47:28.227 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:28.226+0000 7f5358fa8640 1 -- 172.21.15.112:0/1303783622 wait complete. 2024-04-25T16:47:28.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 964394 10.84 MB/sec execute 334 sec latency 56.934 ms 2024-04-25T16:47:29.007 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:28 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setuser ceph since I am not root 2024-04-25T16:47:29.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:28 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setgroup ceph since I am not root 2024-04-25T16:47:29.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:28 smithi116 ceph-mds[62441]: -- 172.21.15.116:0/738910252 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff8d04ba40 con 0x55ff8d093a80 2024-04-25T16:47:29.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:28 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: starting mds.k at 2024-04-25T16:47:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:29 smithi133 ceph-mon[32691]: pgmap v283: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-04-25T16:47:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:29 smithi133 ceph-mon[32691]: daemon mds.k restarted 2024-04-25T16:47:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:29 smithi116 ceph-mon[32712]: pgmap v283: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-04-25T16:47:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:29 smithi116 ceph-mon[32712]: daemon mds.k restarted 2024-04-25T16:47:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:29 smithi112 ceph-mon[29122]: pgmap v283: 97 pgs: 97 active+clean; 80 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 300 op/s 2024-04-25T16:47:29.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:29 smithi112 ceph-mon[29122]: daemon mds.k restarted 2024-04-25T16:47:29.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 966686 10.85 MB/sec execute 335 sec latency 15.255 ms 2024-04-25T16:47:30.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:30 smithi133 ceph-mon[32691]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T16:47:30.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:30 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6832/3273758400,v1:172.21.15.116:6833/3273758400] up:boot 2024-04-25T16:47:30.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:30 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:47:30.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:30 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:30.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:30 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:30.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:30 smithi116 ceph-mon[32712]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T16:47:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:30 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6832/3273758400,v1:172.21.15.116:6833/3273758400] up:boot 2024-04-25T16:47:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:30 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:47:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:30 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:30.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:30 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:30.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:30 smithi112 ceph-mon[29122]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T16:47:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:30 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6832/3273758400,v1:172.21.15.116:6833/3273758400] up:boot 2024-04-25T16:47:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:30 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:47:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:30 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:30.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:30 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:30.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 968520 10.84 MB/sec execute 336 sec latency 62.460 ms 2024-04-25T16:47:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:31 smithi133 ceph-mon[32691]: pgmap v285: 97 pgs: 97 active+clean; 66 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-04-25T16:47:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:31 smithi116 ceph-mon[32712]: pgmap v285: 97 pgs: 97 active+clean; 66 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-04-25T16:47:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:31 smithi112 ceph-mon[29122]: pgmap v285: 97 pgs: 97 active+clean; 66 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-04-25T16:47:31.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 970406 10.84 MB/sec execute 337 sec latency 163.938 ms 2024-04-25T16:47:32.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 972685 10.84 MB/sec execute 338 sec latency 21.466 ms 2024-04-25T16:47:33.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:33 smithi133 ceph-mon[32691]: pgmap v286: 97 pgs: 97 active+clean; 66 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-04-25T16:47:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:33 smithi116 ceph-mon[32712]: pgmap v286: 97 pgs: 97 active+clean; 66 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-04-25T16:47:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:33 smithi112 ceph-mon[29122]: pgmap v286: 97 pgs: 97 active+clean; 66 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 333 op/s 2024-04-25T16:47:33.887 INFO:tasks.workunit.client.0.smithi112.stdout: 1 974623 10.84 MB/sec execute 339 sec latency 15.380 ms 2024-04-25T16:47:34.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 977009 10.86 MB/sec execute 340 sec latency 41.256 ms 2024-04-25T16:47:35.397 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:35 smithi112 ceph-mon[29122]: pgmap v287: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:35 smithi133 ceph-mon[32691]: pgmap v287: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:35.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:35 smithi116 ceph-mon[32712]: pgmap v287: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:35.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 978626 10.84 MB/sec execute 341 sec latency 19.956 ms 2024-04-25T16:47:36.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 980622 10.85 MB/sec execute 342 sec latency 156.516 ms 2024-04-25T16:47:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:37 smithi133 ceph-mon[32691]: pgmap v288: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:37.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:37 smithi116 ceph-mon[32712]: pgmap v288: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:37 smithi112 ceph-mon[29122]: pgmap v288: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:37.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 981721 10.83 MB/sec execute 343 sec latency 55.371 ms 2024-04-25T16:47:38.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 984113 10.84 MB/sec execute 344 sec latency 19.144 ms 2024-04-25T16:47:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:39 smithi133 ceph-mon[32691]: pgmap v289: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:39.509 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:39 smithi116 ceph-mon[32712]: pgmap v289: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:39 smithi112 ceph-mon[29122]: pgmap v289: 97 pgs: 97 active+clean; 72 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-04-25T16:47:39.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 985518 10.83 MB/sec execute 345 sec latency 21.399 ms 2024-04-25T16:47:40.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 987713 10.83 MB/sec execute 346 sec latency 17.205 ms 2024-04-25T16:47:41.328 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 -- 172.21.15.112:0/3620923585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f301020b0 msgr2=0x7f8f301024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3620923585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f301020b0 0x7f8f301024c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009900 tx=0x7f8f2402a410 comp rx=0 tx=0).stop 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 -- 172.21.15.112:0/3620923585 shutdown_connections 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3620923585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f3010e5b0 0x7f8f301109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3620923585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f301020b0 0x7f8f301024c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3620923585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 0x7f8f30101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 -- 172.21.15.112:0/3620923585 >> 172.21.15.112:0/3620923585 conn(0x7f8f300fd8a0 msgr2=0x7f8f300ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 -- 172.21.15.112:0/3620923585 shutdown_connections 2024-04-25T16:47:41.329 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.330+0000 7f8f34df3640 1 -- 172.21.15.112:0/3620923585 wait complete. 2024-04-25T16:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.331+0000 7f8f34df3640 1 Processor -- start 2024-04-25T16:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.331+0000 7f8f34df3640 1 -- start start 2024-04-25T16:47:41.331 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.331+0000 7f8f34df3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f301020b0 0x7f8f3019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.331+0000 7f8f34df3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 0x7f8f3019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f34df3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 0x7f8f3019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f34df3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30113d70 con 0x7f8f3010e5b0 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f34df3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30113bf0 con 0x7f8f301020b0 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f34df3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f30113ef0 con 0x7f8f30109c00 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 0x7f8f3019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 0x7f8f3019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 0x7f8f3019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41302/0 (socket says 172.21.15.112:41302) 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 -- 172.21.15.112:0/3007718971 learned_addr learned my addr 172.21.15.112:0/3007718971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f301020b0 0x7f8f3019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 -- 172.21.15.112:0/3007718971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 msgr2=0x7f8f3019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 0x7f8f3019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.332 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 -- 172.21.15.112:0/3007718971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f301020b0 msgr2=0x7f8f3019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f301020b0 0x7f8f3019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 -- 172.21.15.112:0/3007718971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3019f570 con 0x7f8f3010e5b0 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 0x7f8f3019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f2ed9d640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 0x7f8f3019efb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800eb70 tx=0x7f8f2800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f1f7fe640 1 -- 172.21.15.112:0/3007718971 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28004d20 con 0x7f8f3010e5b0 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3019f860 con 0x7f8f3010e5b0 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.332+0000 7f8f1f7fe640 1 -- 172.21.15.112:0/3007718971 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f2800ce70 con 0x7f8f3010e5b0 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.333+0000 7f8f1f7fe640 1 -- 172.21.15.112:0/3007718971 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f280107f0 con 0x7f8f3010e5b0 2024-04-25T16:47:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.333+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3019fd80 con 0x7f8f3010e5b0 2024-04-25T16:47:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 Processor -- start 2024-04-25T16:47:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f1f7fe640 1 -- 172.21.15.112:0/3007718971 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f8f28010990 con 0x7f8f3010e5b0 2024-04-25T16:47:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 -- start start 2024-04-25T16:47:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 0x561bce193af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce194030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce194570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561bce0fa460 con 0x561bcddf41b0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561bce0b4590 con 0x561bcddcb2f0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561bce0f6420 con 0x561bcddd34d0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 0x561bce193af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 0x561bce193af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41306/0 (socket says 172.21.15.112:41306) 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.336+0000 7f8f2dd9b640 1 -- 172.21.15.112:0/1135347144 learned_addr learned my addr 172.21.15.112:0/1135347144 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce194570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce194030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f1f7fe640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8f180afc50 0x7f8f180b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2ed9d640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8f180afc50 0x7f8f180b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 -- 172.21.15.112:0/1135347144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 msgr2=0x561bce194570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce194570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 -- 172.21.15.112:0/1135347144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 msgr2=0x561bce194030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce194030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 -- 172.21.15.112:0/1135347144 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bce0c10d0 con 0x561bcddf41b0 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f1f7fe640 1 -- 172.21.15.112:0/3007718971 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f8f280df750 con 0x7f8f3010e5b0 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 0x561bce193af0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f3006bf30 tx=0x7f8f24009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8357cbc7e68efc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f2ed9d640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8f180afc50 0x7f8f180b2160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f2801b000 tx=0x7f8f280a3150 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.337+0000 7f8f1cff9640 1 -- 172.21.15.112:0/1135347144 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24035040 con 0x561bcddf41b0 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f1cff9640 1 -- 172.21.15.112:0/1135347144 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f24004380 con 0x561bcddf41b0 2024-04-25T16:47:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f1cff9640 1 -- 172.21.15.112:0/1135347144 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2403b670 con 0x561bcddf41b0 2024-04-25T16:47:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1135347144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 msgr2=0x561bce193af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 0x561bce193af0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8f3006bf30 tx=0x7f8f24009560 comp rx=0 tx=0).stop 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1135347144 shutdown_connections 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce194570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce194030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1135347144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddf41b0 0x561bce193af0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.338+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1135347144 >> 172.21.15.112:0/1135347144 conn(0x561bcdde8300 msgr2=0x561bce1902a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1135347144 shutdown_connections 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1135347144 wait complete. 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 -- start start 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 Processor -- start 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 0x561bce19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce19e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddf41b0 0x561bce1a4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x561bce1a0bc0 con 0x561bcddcb2f0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561bce081b60 con 0x561bcddf41b0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561bce1a0d40 con 0x561bcddd34d0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f3bd7b740 10 client.0 _ll_register_callbacks cb 0x561bce0a62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 0x561bce19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 0x561bce19e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41322/0 (socket says 172.21.15.112:41322) 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.339+0000 7f8f2dd9b640 1 -- 172.21.15.112:0/2611496572 learned_addr learned my addr 172.21.15.112:0/2611496572 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddf41b0 0x561bce1a4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2e59c640 1 -- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddf41b0 msgr2=0x561bce1a4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddf41b0 0x561bce1a4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2e59c640 1 -- 172.21.15.112:0/2611496572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 msgr2=0x561bce19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 0x561bce19e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2e59c640 1 -- 172.21.15.112:0/2611496572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bcde0c210 con 0x561bcddd34d0 2024-04-25T16:47:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 0x561bce19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce19e720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x561bce191230 tx=0x7f8f20010c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eab421588525aff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f15ffb640 1 -- 172.21.15.112:0/2611496572 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20004260 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f15ffb640 1 -- 172.21.15.112:0/2611496572 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f8f20004400 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f24004380 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f15ffb640 1 -- 172.21.15.112:0/2611496572 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8f2000f950 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f3bd7b740 10 client.25780 fetch_fsmap learned FSMap version 31 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bcded8620 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.340+0000 7f8f15ffb640 1 -- 172.21.15.112:0/2611496572 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f24004380 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f15ffb640 1 -- 172.21.15.112:0/2611496572 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f8f20022420 con 0x561bcddd34d0 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f3bd7b740 10 client.25780 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f3bd7b740 10 client.25780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f3bd7b740 20 client.25780 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcddb2d80 0x561bce1b0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f3bd7b740 4 client.25780 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bce108070 con 0x561bcddb2d80 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcddb2d80 0x561bce1b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f15ffb640 10 client.25780 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.341+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcddb2d80 0x561bce1b0820 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce25b9bae50b4f8 server_cookie=dfc2134e7dfe3633 in_seq=0 out_seq=0 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f15ffb640 1 -- 172.21.15.112:0/2611496572 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561bce108070 con 0x561bcddb2d80 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f15ffb640 10 client.25780 handle_command_reply: tid=0 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 client.25780 shutdown 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 msgr2=0x561bce19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce19e720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x561bce191230 tx=0x7f8f20010c60 comp rx=0 tx=0).stop 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 shutdown_connections 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcddb2d80 0x561bce1b0820 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddd34d0 0x561bce19e720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddf41b0 0x561bce1a4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/2611496572 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddcb2f0 0x561bce19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f15ffb640 0 client.25780 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 >> 172.21.15.112:0/2611496572 conn(0x561bcdde8300 msgr2=0x561bce193920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 shutdown_connections 2024-04-25T16:47:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/2611496572 wait complete. 2024-04-25T16:47:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.342+0000 7f8f3bd7b740 20 client.25780 trim_cache size 0 max 16384 2024-04-25T16:47:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 Processor -- start 2024-04-25T16:47:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 -- start start 2024-04-25T16:47:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301b22d0 con 0x561bcddd34d0 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301afba0 con 0x561bcddcb2f0 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301ad9f0 con 0x561bcddf41b0 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.358+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2dd9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41336/0 (socket says 172.21.15.112:41336) 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 -- 172.21.15.112:0/1593374944 learned_addr learned my addr 172.21.15.112:0/1593374944 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:41.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 -- 172.21.15.112:0/1593374944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 msgr2=0x561bce1b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 -- 172.21.15.112:0/1593374944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 msgr2=0x561bce1a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 -- 172.21.15.112:0/1593374944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bce1a4520 con 0x561bcddd34d0 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1b6020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8f200058c0 tx=0x7f8f20022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc16579dcd824d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f17fff640 1 -- 172.21.15.112:0/1593374944 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20022420 con 0x561bcddd34d0 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f17fff640 1 -- 172.21.15.112:0/1593374944 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f200225c0 con 0x561bcddd34d0 2024-04-25T16:47:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.359+0000 7f8f17fff640 1 -- 172.21.15.112:0/1593374944 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200038e0 con 0x561bcddd34d0 2024-04-25T16:47:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1593374944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 msgr2=0x561bce1b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1b6020 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f8f200058c0 tx=0x7f8f20022c90 comp rx=0 tx=0).stop 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1593374944 shutdown_connections 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1b6020 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/1593374944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1593374944 >> 172.21.15.112:0/1593374944 conn(0x561bcdde8300 msgr2=0x561bce19c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1593374944 shutdown_connections 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.360+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/1593374944 wait complete. 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 -- start start 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 Processor -- start 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301ae060 con 0x561bcddd34d0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301af4d0 con 0x561bcddcb2f0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8f301ae730 con 0x561bcddf41b0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f3bd7b740 10 client.0 _ll_register_callbacks cb 0x7f8f2000a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.361+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1a5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41346/0 (socket says 172.21.15.112:41346) 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 -- 172.21.15.112:0/821540924 learned_addr learned my addr 172.21.15.112:0/821540924 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 -- 172.21.15.112:0/821540924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 msgr2=0x561bce1a6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1a6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 -- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 msgr2=0x561bce1aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1aa9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.365 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 -- 172.21.15.112:0/821540924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bce1a67a0 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f2e59c640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1a5cf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8f20022890 tx=0x7f8f2000f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511ca8b91ca35ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f15ffb640 1 -- 172.21.15.112:0/821540924 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20022420 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f15ffb640 1 -- 172.21.15.112:0/821540924 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8556+0+0 (secure 0 0 0) 0x7f8f200054f0 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f200225c0 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f15ffb640 1 -- 172.21.15.112:0/821540924 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8f20043050 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f3bd7b740 10 client.16167 fetch_fsmap learned FSMap version 31 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.362+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561bce2188f0 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f15ffb640 1 -- 172.21.15.112:0/821540924 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f200225c0 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f15ffb640 1 -- 172.21.15.112:0/821540924 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f8f20029070 con 0x561bcddd34d0 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f3bd7b740 10 client.16167 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f3bd7b740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f3bd7b740 20 client.16167 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcdd881c0 0x561bce21afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f3bd7b740 4 client.16167 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561bce101ee0 con 0x561bcdd881c0 2024-04-25T16:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcdd881c0 0x561bce21afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f15ffb640 10 client.16167 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.363+0000 7f8f2dd9b640 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcdd881c0 0x561bce21afe0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4086be691a640e23 server_cookie=9a419fe22bcb88bf in_seq=0 out_seq=0 2024-04-25T16:47:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:41 smithi133 ceph-mon[32691]: pgmap v290: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-04-25T16:47:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:41 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:47:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:41 smithi116 ceph-mon[32712]: pgmap v290: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-04-25T16:47:41.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:41 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.560+0000 7f8f15ffb640 1 -- 172.21.15.112:0/821540924 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561bce101ee0 con 0x561bcdd881c0 2024-04-25T16:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.560+0000 7f8f15ffb640 10 client.16167 handle_command_reply: tid=0 2024-04-25T16:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.560+0000 7f8f3bd7b740 1 client.16167 shutdown 2024-04-25T16:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.560+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 msgr2=0x561bce1a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.560+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1a5cf0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8f20022890 tx=0x7f8f2000f880 comp rx=0 tx=0).stop 2024-04-25T16:47:41.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 shutdown_connections 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x561bcdd881c0 0x561bce21afe0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561bcddf41b0 0x561bce1a6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x561bcddd34d0 0x561bce1a5cf0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f15ffb640 0 client.16167 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 --2- 172.21.15.112:0/821540924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561bcddcb2f0 0x561bce1aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 >> 172.21.15.112:0/821540924 conn(0x561bcdde8300 msgr2=0x561bce193880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 shutdown_connections 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 1 -- 172.21.15.112:0/821540924 wait complete. 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f3bd7b740 20 client.16167 trim_cache size 0 max 16384 2024-04-25T16:47:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:47:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:47:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:47:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:47:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8f180afc50 msgr2=0x7f8f180b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8f180afc50 0x7f8f180b2160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f2801b000 tx=0x7f8f280a3150 comp rx=0 tx=0).stop 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 msgr2=0x7f8f3019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 0x7f8f3019efb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800eb70 tx=0x7f8f2800c6a0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 shutdown_connections 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8f180afc50 0x7f8f180b2160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8f30109c00 0x7f8f3019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f3010e5b0 0x7f8f3019efb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 --2- 172.21.15.112:0/3007718971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f301020b0 0x7f8f3019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 >> 172.21.15.112:0/3007718971 conn(0x7f8f300fd8a0 msgr2=0x7f8f300ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.561+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 shutdown_connections 2024-04-25T16:47:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:41.562+0000 7f8f34df3640 1 -- 172.21.15.112:0/3007718971 wait complete. 2024-04-25T16:47:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:41 smithi112 ceph-mon[29122]: pgmap v290: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 199 op/s 2024-04-25T16:47:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:41 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:47:41.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 988332 10.81 MB/sec execute 347 sec latency 73.464 ms 2024-04-25T16:47:42.007 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:41 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setuser ceph since I am not root 2024-04-25T16:47:42.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:41 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setgroup ceph since I am not root 2024-04-25T16:47:42.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:41 smithi116 ceph-mds[62441]: -- 172.21.15.116:0/1582424422 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cd59e5a40 con 0x560cd5a2d180 2024-04-25T16:47:42.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:41 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: starting mds.k at 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: daemon mds.k restarted 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6832/1410798372,v1:172.21.15.116:6833/1410798372] up:boot 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:42 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: daemon mds.k restarted 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6832/1410798372,v1:172.21.15.116:6833/1410798372] up:boot 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:42.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:42 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:42.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: daemon mds.k restarted 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6832/1410798372,v1:172.21.15.116:6833/1410798372] up:boot 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:42.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:42 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:42.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 990650 10.81 MB/sec execute 348 sec latency 15.100 ms 2024-04-25T16:47:43.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:43 smithi133 ceph-mon[32691]: pgmap v291: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 175 op/s 2024-04-25T16:47:43.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:43 smithi116 ceph-mon[32712]: pgmap v291: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 175 op/s 2024-04-25T16:47:43.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:43 smithi112 ceph-mon[29122]: pgmap v291: 97 pgs: 97 active+clean; 98 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 175 op/s 2024-04-25T16:47:43.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 991836 10.80 MB/sec execute 349 sec latency 33.493 ms 2024-04-25T16:47:44.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 993813 10.79 MB/sec execute 350 sec latency 43.656 ms 2024-04-25T16:47:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:45 smithi133 ceph-mon[32691]: pgmap v293: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:45.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:45 smithi116 ceph-mon[32712]: pgmap v293: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:45.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:45 smithi112 ceph-mon[29122]: pgmap v293: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:45.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 995350 10.79 MB/sec execute 351 sec latency 15.874 ms 2024-04-25T16:47:46.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 997204 10.78 MB/sec execute 352 sec latency 96.302 ms 2024-04-25T16:47:47.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:47 smithi133 ceph-mon[32691]: pgmap v294: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:47.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:47 smithi116 ceph-mon[32712]: pgmap v294: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:47.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:47 smithi112 ceph-mon[29122]: pgmap v294: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:47.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 998753 10.79 MB/sec execute 353 sec latency 15.972 ms 2024-04-25T16:47:48.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1000778 10.78 MB/sec execute 354 sec latency 17.190 ms 2024-04-25T16:47:49.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:49 smithi133 ceph-mon[32691]: pgmap v295: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:49 smithi116 ceph-mon[32712]: pgmap v295: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:49.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:49 smithi112 ceph-mon[29122]: pgmap v295: 97 pgs: 97 active+clean; 88 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 325 op/s 2024-04-25T16:47:49.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1002418 10.78 MB/sec execute 355 sec latency 15.467 ms 2024-04-25T16:47:50.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1004765 10.78 MB/sec execute 356 sec latency 16.844 ms 2024-04-25T16:47:51.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:51 smithi133 ceph-mon[32691]: pgmap v296: 97 pgs: 97 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-04-25T16:47:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:51 smithi116 ceph-mon[32712]: pgmap v296: 97 pgs: 97 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-04-25T16:47:51.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:51 smithi112 ceph-mon[29122]: pgmap v296: 97 pgs: 97 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-04-25T16:47:51.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1005798 10.77 MB/sec execute 357 sec latency 131.230 ms 2024-04-25T16:47:52.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:52 smithi116 ceph-mon[32712]: pgmap v297: 97 pgs: 97 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-04-25T16:47:52.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:52 smithi112 ceph-mon[29122]: pgmap v297: 97 pgs: 97 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-04-25T16:47:52.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1007738 10.76 MB/sec execute 358 sec latency 23.175 ms 2024-04-25T16:47:52.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:52 smithi133 ceph-mon[32691]: pgmap v297: 97 pgs: 97 active+clean; 94 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-04-25T16:47:53.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1009226 10.76 MB/sec execute 359 sec latency 120.216 ms 2024-04-25T16:47:54.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 -- 172.21.15.112:0/1049989318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 msgr2=0x7fc264106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/1049989318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 0x7fc264106c30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc254009900 tx=0x7fc25402a410 comp rx=0 tx=0).stop 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 -- 172.21.15.112:0/1049989318 shutdown_connections 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/1049989318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 0x7fc264106c30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/1049989318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2641072c0 0x7fc26410ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/1049989318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc264102270 0x7fc264102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 -- 172.21.15.112:0/1049989318 >> 172.21.15.112:0/1049989318 conn(0x7fc2640fd830 msgr2=0x7fc2640ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 -- 172.21.15.112:0/1049989318 shutdown_connections 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 -- 172.21.15.112:0/1049989318 wait complete. 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.663+0000 7fc26ad9d640 1 Processor -- start 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 -- start start 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 0x7fc26419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 0x7fc26419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2641072c0 0x7fc26419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc264113b60 con 0x7fc264102270 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641139e0 con 0x7fc2641072c0 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26ad9d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc264113ce0 con 0x7fc264106800 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc269d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 0x7fc26419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2641072c0 0x7fc26419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc269d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 0x7fc26419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57394/0 (socket says 172.21.15.112:57394) 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc269d9b640 1 -- 172.21.15.112:0/752174815 learned_addr learned my addr 172.21.15.112:0/752174815 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26959a640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 0x7fc26419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 -- 172.21.15.112:0/752174815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 msgr2=0x7fc26419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 0x7fc26419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 -- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 msgr2=0x7fc26419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 0x7fc26419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 -- 172.21.15.112:0/752174815 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26419f400 con 0x7fc2641072c0 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc269d9b640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 0x7fc26419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.664+0000 7fc26a59c640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2641072c0 0x7fc26419edc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00e970 tx=0x7fc25c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.665+0000 7fc25affd640 1 -- 172.21.15.112:0/752174815 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00ccc0 con 0x7fc2641072c0 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.665+0000 7fc25affd640 1 -- 172.21.15.112:0/752174815 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c0035f0 con 0x7fc2641072c0 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.665+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26419f6f0 con 0x7fc2641072c0 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.665+0000 7fc25affd640 1 -- 172.21.15.112:0/752174815 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c00ce60 con 0x7fc2641072c0 2024-04-25T16:47:54.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.665+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26419fbc0 con 0x7fc2641072c0 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.667+0000 7fc25affd640 1 -- 172.21.15.112:0/752174815 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fc25c004020 con 0x7fc2641072c0 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.667+0000 7fc272f87740 1 Processor -- start 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 -- start start 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 0x55c26e457b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e08f2f0 0x55c26e458060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e4585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.675 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c26e3be460 con 0x55c26e08f2f0 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c26e378590 con 0x55c26e0b81b0 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc272f87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c26e3ba420 con 0x55c26e0974d0 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc269d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e08f2f0 0x55c26e458060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 0x55c26e457b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 0x55c26e457b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:58096/0 (socket says 172.21.15.112:58096) 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc269d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e08f2f0 0x55c26e458060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57398/0 (socket says 172.21.15.112:57398) 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 -- 172.21.15.112:0/2337289944 learned_addr learned my addr 172.21.15.112:0/2337289944 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e4585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc25affd640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc2500afc50 0x7fc2500b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 -- 172.21.15.112:0/2337289944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 msgr2=0x55c26e4585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e4585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 -- 172.21.15.112:0/2337289944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e08f2f0 msgr2=0x55c26e458060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e08f2f0 0x55c26e458060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.676 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26959a640 1 -- 172.21.15.112:0/2337289944 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26e3850d0 con 0x55c26e0b81b0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.668+0000 7fc26a59c640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc2500afc50 0x7fc2500b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.669+0000 7fc25affd640 1 -- 172.21.15.112:0/752174815 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fc25c0df1f0 con 0x7fc2641072c0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.669+0000 7fc26959a640 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 0x55c26e457b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc264075340 tx=0x7fc254005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f032ba9b644159c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.669+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337289944 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254035030 con 0x55c26e0b81b0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.669+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337289944 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2540035f0 con 0x55c26e0b81b0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.669+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337289944 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2540044e0 con 0x55c26e0b81b0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 -- 172.21.15.112:0/2337289944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 msgr2=0x55c26e457b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 0x55c26e457b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc264075340 tx=0x7fc254005780 comp rx=0 tx=0).stop 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc26a59c640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc2500afc50 0x7fc2500b2160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0a7300 tx=0x7fc25c0045a0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 -- 172.21.15.112:0/2337289944 shutdown_connections 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e4585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e08f2f0 0x55c26e458060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 --2- 172.21.15.112:0/2337289944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e0b81b0 0x55c26e457b20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 -- 172.21.15.112:0/2337289944 >> 172.21.15.112:0/2337289944 conn(0x55c26e0ac300 msgr2=0x55c26e454440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 -- 172.21.15.112:0/2337289944 shutdown_connections 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 -- 172.21.15.112:0/2337289944 wait complete. 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 -- start start 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.670+0000 7fc272f87740 1 Processor -- start 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e08f2f0 0x55c26e468100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e468640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 0x55c26e4665c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55c26e4691d0 con 0x55c26e0b81b0 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c26e345b60 con 0x55c26e08f2f0 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c26e469350 con 0x55c26e0974d0 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc272f87740 10 client.0 _ll_register_callbacks cb 0x55c26e36a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 0x55c26e4665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 0x55c26e4665c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57412/0 (socket says 172.21.15.112:57412) 2024-04-25T16:47:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 -- 172.21.15.112:0/4150044802 learned_addr learned my addr 172.21.15.112:0/4150044802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e08f2f0 0x55c26e468100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc269d9b640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e468640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 -- 172.21.15.112:0/4150044802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 msgr2=0x55c26e468640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e468640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 -- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e08f2f0 msgr2=0x55c26e468100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e08f2f0 0x55c26e468100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.671+0000 7fc26959a640 1 -- 172.21.15.112:0/4150044802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c26e0d0210 con 0x55c26e0b81b0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc26959a640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 0x55c26e4665c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55c26e465e20 tx=0x7fc254032dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa860c58e313d8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc24d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc24d7fa640 1 -- 172.21.15.112:0/4150044802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254035040 con 0x55c26e0b81b0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc24d7fa640 1 -- 172.21.15.112:0/4150044802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fc2540039a0 con 0x55c26e0b81b0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2540035f0 con 0x55c26e0b81b0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc24d7fa640 1 -- 172.21.15.112:0/4150044802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc254003ba0 con 0x55c26e0b81b0 2024-04-25T16:47:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 10 client.16182 fetch_fsmap learned FSMap version 32 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c26e19c620 con 0x55c26e0b81b0 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc24d7fa640 1 -- 172.21.15.112:0/4150044802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2540035f0 con 0x55c26e0b81b0 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc24d7fa640 1 -- 172.21.15.112:0/4150044802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc25404a980 con 0x55c26e0b81b0 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 10 client.16182 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 20 client.16182 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e076d80 0x55c26e478b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 4 client.16182 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.672+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c26e3cc070 con 0x55c26e076d80 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.673+0000 7fc269d9b640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e076d80 0x55c26e478b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.673+0000 7fc24d7fa640 10 client.16182 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.673+0000 7fc269d9b640 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e076d80 0x55c26e478b40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bdd8b7f6c6df62f server_cookie=43d564024ddd3ac1 in_seq=0 out_seq=0 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc24d7fa640 1 -- 172.21.15.112:0/4150044802 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c26e3cc070 con 0x55c26e076d80 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc24d7fa640 10 client.16182 handle_command_reply: tid=0 2024-04-25T16:47:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 client.16182 shutdown 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 msgr2=0x55c26e4665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 0x55c26e4665c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55c26e465e20 tx=0x7fc254032dc0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 shutdown_connections 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e076d80 0x55c26e478b40 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e0974d0 0x55c26e468640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0b81b0 0x55c26e4665c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 --2- 172.21.15.112:0/4150044802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e08f2f0 0x55c26e468100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc24d7fa640 0 client.16182 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 >> 172.21.15.112:0/4150044802 conn(0x55c26e0ac300 msgr2=0x55c26e4584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 shutdown_connections 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 1 -- 172.21.15.112:0/4150044802 wait complete. 2024-04-25T16:47:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.675+0000 7fc272f87740 20 client.16182 trim_cache size 0 max 16384 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 Processor -- start 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 -- start start 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e45fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e460410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e463800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641b0020 con 0x55c26e0974d0 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ad8f0 con 0x55c26e076d80 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc272f87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ab740 con 0x55c26e08f2f0 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e463800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e45fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc269d9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e460410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e45fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:58106/0 (socket says 172.21.15.112:58106) 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 -- 172.21.15.112:0/3638245961 learned_addr learned my addr 172.21.15.112:0/3638245961 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 -- 172.21.15.112:0/3638245961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 msgr2=0x55c26e460410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e460410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 -- 172.21.15.112:0/3638245961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 msgr2=0x55c26e45fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e45fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc26959a640 1 -- 172.21.15.112:0/3638245961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26e459020 con 0x55c26e0974d0 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.691+0000 7fc269d9b640 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e460410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.692+0000 7fc26959a640 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e463800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc25404aea0 tx=0x7fc254009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb77591123173594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.692+0000 7fc24f7fe640 1 -- 172.21.15.112:0/3638245961 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254035040 con 0x55c26e0974d0 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.692+0000 7fc24f7fe640 1 -- 172.21.15.112:0/3638245961 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc254003f90 con 0x55c26e0974d0 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.692+0000 7fc24f7fe640 1 -- 172.21.15.112:0/3638245961 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254005240 con 0x55c26e0974d0 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 -- 172.21.15.112:0/3638245961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 msgr2=0x55c26e463800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e463800 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc25404aea0 tx=0x7fc254009640 comp rx=0 tx=0).stop 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 -- 172.21.15.112:0/3638245961 shutdown_connections 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e460410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e463800 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 --2- 172.21.15.112:0/3638245961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e45fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 -- 172.21.15.112:0/3638245961 >> 172.21.15.112:0/3638245961 conn(0x55c26e0ac300 msgr2=0x55c26e4619f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 -- 172.21.15.112:0/3638245961 shutdown_connections 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 -- 172.21.15.112:0/3638245961 wait complete. 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 -- start start 2024-04-25T16:47:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.693+0000 7fc272f87740 1 Processor -- start 2024-04-25T16:47:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e47be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e4791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e479720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641abdb0 con 0x55c26e0974d0 2024-04-25T16:47:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ad220 con 0x55c26e076d80 2024-04-25T16:47:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2641ac480 con 0x55c26e08f2f0 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 10 client.0 _ll_register_callbacks cb 0x7fc254009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e479720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e47be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e47be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:58120/0 (socket says 172.21.15.112:58120) 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 -- 172.21.15.112:0/2833319773 learned_addr learned my addr 172.21.15.112:0/2833319773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc269d9b640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e4791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 -- 172.21.15.112:0/2833319773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 msgr2=0x55c26e4791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e4791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 -- 172.21.15.112:0/2833319773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 msgr2=0x55c26e479720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e479720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 -- 172.21.15.112:0/2833319773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c26e479c60 con 0x55c26e076d80 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc269d9b640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e4791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:47:54.697 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc26959a640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e47be20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55c26e467710 tx=0x7fc254005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b5ed9e872ee5b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc24d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc24d7fa640 1 -- 172.21.15.112:0/2833319773 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2540037f0 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc254003f90 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.694+0000 7fc24d7fa640 1 -- 172.21.15.112:0/2833319773 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8556+0+0 (secure 0 0 0) 0x7fc254003990 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc24d7fa640 1 -- 172.21.15.112:0/2833319773 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc254005030 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 10 client.25889 fetch_fsmap learned FSMap version 32 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c26e4e08a0 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc24d7fa640 1 -- 172.21.15.112:0/2833319773 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc254003f90 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc24d7fa640 1 -- 172.21.15.112:0/2833319773 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fc25404a9b0 con 0x55c26e076d80 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 10 client.25889 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 10 client.25889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 20 client.25889 populate_metadata read hostname 'smithi112' 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e0b81b0 0x55c26e4e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:47:54.698 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 4 client.25889 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:47:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c26e3c5ee0 con 0x55c26e0b81b0 2024-04-25T16:47:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.695+0000 7fc269d9b640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e0b81b0 0x55c26e4e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:47:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.696+0000 7fc24d7fa640 10 client.25889 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:54.699 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.696+0000 7fc269d9b640 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e0b81b0 0x55c26e4e2eb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a59299d5e13555c server_cookie=5131d9137a7f1405 in_seq=0 out_seq=0 2024-04-25T16:47:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.861+0000 7fc24d7fa640 1 -- 172.21.15.112:0/2833319773 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c26e3c5ee0 con 0x55c26e0b81b0 2024-04-25T16:47:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.861+0000 7fc24d7fa640 10 client.25889 handle_command_reply: tid=0 2024-04-25T16:47:54.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.861+0000 7fc272f87740 1 client.25889 shutdown 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 msgr2=0x55c26e47be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e47be20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55c26e467710 tx=0x7fc254005fd0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 shutdown_connections 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55c26e0b81b0 0x55c26e4e2eb0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c26e08f2f0 0x55c26e4791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55c26e0974d0 0x55c26e479720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 --2- 172.21.15.112:0/2833319773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c26e076d80 0x55c26e47be20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc24d7fa640 0 client.25889 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 >> 172.21.15.112:0/2833319773 conn(0x55c26e0ac300 msgr2=0x55c26e467a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 shutdown_connections 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 1 -- 172.21.15.112:0/2833319773 wait complete. 2024-04-25T16:47:54.861 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.862+0000 7fc272f87740 20 client.25889 trim_cache size 0 max 16384 2024-04-25T16:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:47:54.862 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc2500afc50 msgr2=0x7fc2500b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc2500afc50 0x7fc2500b2160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc25c0a7300 tx=0x7fc25c0045a0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2641072c0 msgr2=0x7fc26419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2641072c0 0x7fc26419edc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00e970 tx=0x7fc25c00ee30 comp rx=0 tx=0).stop 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 shutdown_connections 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc2500afc50 0x7fc2500b2160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc264106800 0x7fc26419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2641072c0 0x7fc26419edc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 --2- 172.21.15.112:0/752174815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc264102270 0x7fc26419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 >> 172.21.15.112:0/752174815 conn(0x7fc2640fd830 msgr2=0x7fc26410d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 shutdown_connections 2024-04-25T16:47:54.863 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:47:54.863+0000 7fc26ad9d640 1 -- 172.21.15.112:0/752174815 wait complete. 2024-04-25T16:47:54.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1010519 10.75 MB/sec execute 360 sec latency 83.325 ms 2024-04-25T16:47:55.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:54 smithi116 ceph-mon[32712]: pgmap v298: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 366 op/s 2024-04-25T16:47:55.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:54 smithi112 ceph-mon[29122]: pgmap v298: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 366 op/s 2024-04-25T16:47:55.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:54 smithi133 ceph-mon[32691]: pgmap v298: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 366 op/s 2024-04-25T16:47:55.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1012641 10.76 MB/sec execute 361 sec latency 18.634 ms 2024-04-25T16:47:56.007 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setuser ceph since I am not root 2024-04-25T16:47:56.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setgroup ceph since I am not root 2024-04-25T16:47:56.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:55 smithi116 ceph-mds[62441]: -- 172.21.15.116:0/1829759857 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5612620c5a40 con 0x56126210d600 2024-04-25T16:47:56.008 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:47:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: starting mds.k at 2024-04-25T16:47:56.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1013929 10.74 MB/sec execute 362 sec latency 49.190 ms 2024-04-25T16:47:57.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:56 smithi116 ceph-mon[32712]: pgmap v299: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 164 op/s 2024-04-25T16:47:57.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:56 smithi116 ceph-mon[32712]: daemon mds.k restarted 2024-04-25T16:47:57.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:56 smithi112 ceph-mon[29122]: pgmap v299: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 164 op/s 2024-04-25T16:47:57.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:56 smithi112 ceph-mon[29122]: daemon mds.k restarted 2024-04-25T16:47:57.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:56 smithi133 ceph-mon[32691]: pgmap v299: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 164 op/s 2024-04-25T16:47:57.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:56 smithi133 ceph-mon[32691]: daemon mds.k restarted 2024-04-25T16:47:57.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1016056 10.75 MB/sec execute 363 sec latency 35.525 ms 2024-04-25T16:47:58.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:57 smithi116 ceph-mon[32712]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T16:47:58.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:57 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6832/3671227277,v1:172.21.15.116:6833/3671227277] up:boot 2024-04-25T16:47:58.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:57 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:47:58.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:57 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:58.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:57 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:58.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:57 smithi112 ceph-mon[29122]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T16:47:58.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:57 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6832/3671227277,v1:172.21.15.116:6833/3671227277] up:boot 2024-04-25T16:47:58.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:57 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:47:58.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:57 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:58.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:57 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:58.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:57 smithi133 ceph-mon[32691]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T16:47:58.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:57 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6832/3671227277,v1:172.21.15.116:6833/3671227277] up:boot 2024-04-25T16:47:58.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:57 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:47:58.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:57 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:47:58.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:57 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:47:58.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1017232 10.73 MB/sec execute 364 sec latency 69.017 ms 2024-04-25T16:47:59.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:47:58 smithi116 ceph-mon[32712]: pgmap v301: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-04-25T16:47:59.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:47:58 smithi112 ceph-mon[29122]: pgmap v301: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-04-25T16:47:59.099 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:47:58 smithi133 ceph-mon[32691]: pgmap v301: 97 pgs: 97 active+clean; 100 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-04-25T16:47:59.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1019605 10.74 MB/sec execute 365 sec latency 18.815 ms 2024-04-25T16:48:00.888 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1020614 10.72 MB/sec execute 366 sec latency 28.513 ms 2024-04-25T16:48:01.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:00 smithi116 ceph-mon[32712]: pgmap v302: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 306 op/s 2024-04-25T16:48:01.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:00 smithi112 ceph-mon[29122]: pgmap v302: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 306 op/s 2024-04-25T16:48:01.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:00 smithi133 ceph-mon[32691]: pgmap v302: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 306 op/s 2024-04-25T16:48:01.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1022000 10.71 MB/sec execute 367 sec latency 167.802 ms 2024-04-25T16:48:02.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1023558 10.71 MB/sec execute 368 sec latency 17.992 ms 2024-04-25T16:48:03.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:02 smithi116 ceph-mon[32712]: pgmap v303: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 306 op/s 2024-04-25T16:48:03.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:02 smithi112 ceph-mon[29122]: pgmap v303: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 306 op/s 2024-04-25T16:48:03.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:02 smithi133 ceph-mon[32691]: pgmap v303: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 306 op/s 2024-04-25T16:48:03.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1025540 10.70 MB/sec execute 369 sec latency 19.323 ms 2024-04-25T16:48:04.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1027095 10.70 MB/sec execute 370 sec latency 15.341 ms 2024-04-25T16:48:05.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:04 smithi116 ceph-mon[32712]: pgmap v304: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:05.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:04 smithi112 ceph-mon[29122]: pgmap v304: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:05.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:04 smithi133 ceph-mon[32691]: pgmap v304: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:05.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1028603 10.69 MB/sec execute 371 sec latency 51.890 ms 2024-04-25T16:48:06.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1029283 10.67 MB/sec execute 372 sec latency 148.716 ms 2024-04-25T16:48:07.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:06 smithi116 ceph-mon[32712]: pgmap v305: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:07.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:06 smithi116 ceph-mon[32712]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T16:48:07.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:06 smithi112 ceph-mon[29122]: pgmap v305: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:07.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:06 smithi112 ceph-mon[29122]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T16:48:07.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:06 smithi133 ceph-mon[32691]: pgmap v305: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:07.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:06 smithi133 ceph-mon[32691]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T16:48:07.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1030213 10.66 MB/sec execute 373 sec latency 181.412 ms 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.011+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/2218726911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 msgr2=0x7f9a98103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.011+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/2218726911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98103c80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009900 tx=0x7f9a8c02a410 comp rx=0 tx=0).stop 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.011+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/2218726911 shutdown_connections 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.012+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/2218726911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a981042f0 0x7f9a981106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.012+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/2218726911 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98103c80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.012+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/2218726911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a98109ba0 0x7f9a98103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.012+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/2218726911 >> 172.21.15.112:0/2218726911 conn(0x7f9a980792a0 msgr2=0x7f9a980796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.012+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/2218726911 shutdown_connections 2024-04-25T16:48:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.012+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/2218726911 wait complete. 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.013+0000 7f9a9ed0d640 1 Processor -- start 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.013+0000 7f9a9ed0d640 1 -- start start 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9ed0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a981042f0 0x7f9a98074ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9ed0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a98109ba0 0x7f9a980753e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9ed0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9ed0d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98113e40 con 0x7f9a9810a040 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9ed0d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98113cc0 con 0x7f9a98109ba0 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9ed0d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a98113fc0 con 0x7f9a981042f0 2024-04-25T16:48:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9d2aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9d2aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98075920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43618/0 (socket says 172.21.15.112:43618) 2024-04-25T16:48:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9d2aa640 1 -- 172.21.15.112:0/1869322890 learned_addr learned my addr 172.21.15.112:0/1869322890 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a981042f0 0x7f9a98074ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9d2aa640 1 -- 172.21.15.112:0/1869322890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a981042f0 msgr2=0x7f9a98074ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a981042f0 0x7f9a98074ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.014+0000 7f9a9d2aa640 1 -- 172.21.15.112:0/1869322890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a98109ba0 msgr2=0x7f9a980753e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.015+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a98109ba0 0x7f9a980753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.015+0000 7f9a9d2aa640 1 -- 172.21.15.112:0/1869322890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a981a8ae0 con 0x7f9a9810a040 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.015+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98075920 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000eaa0 tx=0x7f9a9000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.015+0000 7f9a95ffb640 1 -- 172.21.15.112:0/1869322890 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a90004d20 con 0x7f9a9810a040 2024-04-25T16:48:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.015+0000 7f9a95ffb640 1 -- 172.21.15.112:0/1869322890 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a90005020 con 0x7f9a9810a040 2024-04-25T16:48:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.016+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a981a8dd0 con 0x7f9a9810a040 2024-04-25T16:48:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.016+0000 7f9a95ffb640 1 -- 172.21.15.112:0/1869322890 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a900116b0 con 0x7f9a9810a040 2024-04-25T16:48:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.016+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a981a9290 con 0x7f9a9810a040 2024-04-25T16:48:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.020+0000 7f9aa5bbf740 1 Processor -- start 2024-04-25T16:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.021+0000 7f9a95ffb640 1 -- 172.21.15.112:0/1869322890 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f9a900051d0 con 0x7f9a9810a040 2024-04-25T16:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.022+0000 7f9a95ffb640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9a800afda0 0x7f9a800b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.022+0000 7f9a95ffb640 1 -- 172.21.15.112:0/1869322890 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8644+0+0 (secure 0 0 0) 0x7f9a900df350 con 0x7f9a9810a040 2024-04-25T16:48:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.022+0000 7f9a97fff640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9a800afda0 0x7f9a800b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.022+0000 7f9aa5bbf740 1 -- start start 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ea59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 0x562533ea5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533ae54d0 0x562533ea6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562533e0c460 con 0x562533b061b0 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562533dc6590 con 0x562533ae54d0 2024-04-25T16:48:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562533e08420 con 0x562533add2f0 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9a97fff640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9a800afda0 0x7f9a800b22b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0096c0 tx=0x7f9a8c031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 0x562533ea5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 0x562533ea5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:41592/0 (socket says 172.21.15.112:41592) 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.023+0000 7f9a97fff640 1 -- 172.21.15.112:0/3775122032 learned_addr learned my addr 172.21.15.112:0/3775122032 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.024+0000 7f9a97fff640 1 -- 172.21.15.112:0/3775122032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533ae54d0 msgr2=0x562533ea6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.024+0000 7f9a97fff640 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533ae54d0 0x562533ea6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.024+0000 7f9a97fff640 1 -- 172.21.15.112:0/3775122032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 msgr2=0x562533ea59e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.024+0000 7f9a97fff640 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ea59e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.024+0000 7f9a97fff640 1 -- 172.21.15.112:0/3775122032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562533dd30d0 con 0x562533add2f0 2024-04-25T16:48:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.027+0000 7f9a97fff640 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 0x562533ea5f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c033000 tx=0x7f9a8c03a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14bb83baf29d1bf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.028+0000 7f9a7effd640 1 -- 172.21.15.112:0/3775122032 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c035040 con 0x562533add2f0 2024-04-25T16:48:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.028+0000 7f9a7effd640 1 -- 172.21.15.112:0/3775122032 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a8c04c070 con 0x562533add2f0 2024-04-25T16:48:08.026 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.028+0000 7f9a7effd640 1 -- 172.21.15.112:0/3775122032 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c047420 con 0x562533add2f0 2024-04-25T16:48:08.027 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.029+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/3775122032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 msgr2=0x562533ea5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.029+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 0x562533ea5f20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c033000 tx=0x7f9a8c03a000 comp rx=0 tx=0).stop 2024-04-25T16:48:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/3775122032 shutdown_connections 2024-04-25T16:48:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533add2f0 0x562533ea5f20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533ae54d0 0x562533ea6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.028 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/3775122032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ea59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/3775122032 >> 172.21.15.112:0/3775122032 conn(0x562533afa300 msgr2=0x562533ea22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/3775122032 shutdown_connections 2024-04-25T16:48:08.029 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.030+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/3775122032 wait complete. 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.031+0000 7f9aa5bbf740 1 -- start start 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.031+0000 7f9aa5bbf740 1 Processor -- start 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 0x562533eba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eba9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533b061b0 0x562533eb88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562533ebb3d0 con 0x562533add2f0 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562533d93b60 con 0x562533b061b0 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562533ebb550 con 0x562533ae54d0 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9aa5bbf740 10 client.0 _ll_register_callbacks cb 0x562533db82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:08.030 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9a9caa9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 0x562533eba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9a9caa9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 0x562533eba460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43632/0 (socket says 172.21.15.112:43632) 2024-04-25T16:48:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9a9caa9640 1 -- 172.21.15.112:0/55171616 learned_addr learned my addr 172.21.15.112:0/55171616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:08.031 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9a97fff640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.032+0000 7f9a9caa9640 1 -- 172.21.15.112:0/55171616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 msgr2=0x562533eba9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eba9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533b061b0 0x562533eb88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9caa9640 1 -- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533b061b0 msgr2=0x562533eb88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533b061b0 0x562533eb88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.032 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9caa9640 1 -- 172.21.15.112:0/55171616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562533b1e210 con 0x562533add2f0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533b061b0 0x562533eb88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 0x562533eba460 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e480 tx=0x7f9a8800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe555a0131be9ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a77fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a77fff640 1 -- 172.21.15.112:0/55171616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88017070 con 0x562533add2f0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a8c04c070 con 0x562533add2f0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a77fff640 1 -- 172.21.15.112:0/55171616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8644+0+0 (secure 0 0 0) 0x7f9a88013680 con 0x562533add2f0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9a77fff640 1 -- 172.21.15.112:0/55171616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9a88021050 con 0x562533add2f0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9aa5bbf740 10 client.16203 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.033+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562533bea620 con 0x562533add2f0 2024-04-25T16:48:08.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.034+0000 7f9a77fff640 1 -- 172.21.15.112:0/55171616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a8c04c070 con 0x562533add2f0 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9a77fff640 1 -- 172.21.15.112:0/55171616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f9a88024020 con 0x562533add2f0 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9aa5bbf740 10 client.16203 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9aa5bbf740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9aa5bbf740 20 client.16203 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533ac4d80 0x562533ecb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9aa5bbf740 4 client.16203 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:48:08.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.035+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562533e1a070 con 0x562533ac4d80 2024-04-25T16:48:08.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.037+0000 7f9a97fff640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533ac4d80 0x562533ecb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.038+0000 7f9a97fff640 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533ac4d80 0x562533ecb0a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9bb6817d69d59d server_cookie=4cd3daf09e47b06c in_seq=0 out_seq=0 2024-04-25T16:48:08.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.038+0000 7f9a77fff640 10 client.16203 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.041+0000 7f9a77fff640 1 -- 172.21.15.112:0/55171616 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562533e1a070 con 0x562533ac4d80 2024-04-25T16:48:08.038 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.041+0000 7f9a77fff640 10 client.16203 handle_command_reply: tid=0 2024-04-25T16:48:08.039 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.041+0000 7f9aa5bbf740 1 client.16203 shutdown 2024-04-25T16:48:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 msgr2=0x562533eba460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 0x562533eba460 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e480 tx=0x7f9a8800e940 comp rx=0 tx=0).stop 2024-04-25T16:48:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 shutdown_connections 2024-04-25T16:48:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533ac4d80 0x562533ecb0a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eba9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.040 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533b061b0 0x562533eb88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/55171616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533add2f0 0x562533eba460 secure :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9a8800e480 tx=0x7f9a8800e940 comp rx=0 tx=0).stop 2024-04-25T16:48:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 >> 172.21.15.112:0/55171616 conn(0x562533afa300 msgr2=0x562533ea77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.042+0000 7f9a77fff640 0 client.16203 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.043+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 shutdown_connections 2024-04-25T16:48:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.043+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/55171616 wait complete. 2024-04-25T16:48:08.041 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.043+0000 7f9aa5bbf740 20 client.16203 trim_cache size 0 max 16384 2024-04-25T16:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 Processor -- start 2024-04-25T16:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 -- start start 2024-04-25T16:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eb7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533eb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533eb2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981b02a0 con 0x562533b061b0 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981adb70 con 0x562533a9a1c0 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981aba40 con 0x562533ae54d0 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533eb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9a9caa9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eb7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533eb88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43640/0 (socket says 172.21.15.112:43640) 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.071+0000 7f9a97fff640 1 -- 172.21.15.112:0/1385083783 learned_addr learned my addr 172.21.15.112:0/1385083783 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533eb2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9caa9640 1 -- 172.21.15.112:0/1385083783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 msgr2=0x562533eb2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533eb2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9caa9640 1 -- 172.21.15.112:0/1385083783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 msgr2=0x562533eb88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533eb88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9caa9640 1 -- 172.21.15.112:0/1385083783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562533ea2e90 con 0x562533ae54d0 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a97fff640 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533eb88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eb7820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9a88008000 tx=0x7f9a88009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=393295df6b516f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:08.071 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.072+0000 7f9a7e7fc640 1 -- 172.21.15.112:0/1385083783 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88017040 con 0x562533ae54d0 2024-04-25T16:48:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.073+0000 7f9a7e7fc640 1 -- 172.21.15.112:0/1385083783 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a880059c0 con 0x562533ae54d0 2024-04-25T16:48:08.072 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.073+0000 7f9a7e7fc640 1 -- 172.21.15.112:0/1385083783 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a88000e60 con 0x562533ae54d0 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/1385083783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 msgr2=0x562533eb7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eb7820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9a88008000 tx=0x7f9a88009f80 comp rx=0 tx=0).stop 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/1385083783 shutdown_connections 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533eb2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533eb88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/1385083783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533eb7820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/1385083783 >> 172.21.15.112:0/1385083783 conn(0x562533afa300 msgr2=0x562533eb81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/1385083783 shutdown_connections 2024-04-25T16:48:08.073 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.074+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/1385083783 wait complete. 2024-04-25T16:48:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.075+0000 7f9aa5bbf740 1 -- start start 2024-04-25T16:48:08.074 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.075+0000 7f9aa5bbf740 1 Processor -- start 2024-04-25T16:48:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533ec1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533ec1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ec24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ac090 con 0x562533b061b0 2024-04-25T16:48:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ad4a0 con 0x562533a9a1c0 2024-04-25T16:48:08.075 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a981ac700 con 0x562533ae54d0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9aa5bbf740 10 client.0 _ll_register_callbacks cb 0x7f9a88009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533ec1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a9d2aa640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ec24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533ec1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:41626/0 (socket says 172.21.15.112:41626) 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 -- 172.21.15.112:0/2931798728 learned_addr learned my addr 172.21.15.112:0/2931798728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a9d2aa640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ec24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43654/0 (socket says 172.21.15.112:43654) 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533ec1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 -- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 msgr2=0x562533ec1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533ec1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 -- 172.21.15.112:0/2931798728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 msgr2=0x562533ec24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ec24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.076+0000 7f9a97fff640 1 -- 172.21.15.112:0/2931798728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562533f31450 con 0x562533ae54d0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.077+0000 7f9a97fff640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533ec1f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009900 tx=0x7f9a8c003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e83b7ec99338d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:08.076 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.077+0000 7f9a77fff640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:48:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.077+0000 7f9a77fff640 1 -- 172.21.15.112:0/2931798728 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c035040 con 0x562533ae54d0 2024-04-25T16:48:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.077+0000 7f9a77fff640 1 -- 172.21.15.112:0/2931798728 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8644+0+0 (secure 0 0 0) 0x7f9a8c003d30 con 0x562533ae54d0 2024-04-25T16:48:08.077 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.077+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a880059c0 con 0x562533ae54d0 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.078+0000 7f9a77fff640 1 -- 172.21.15.112:0/2931798728 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9a8c0318a0 con 0x562533ae54d0 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.078+0000 7f9aa5bbf740 10 client.25831 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.078+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562533f32640 con 0x562533ae54d0 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9a77fff640 1 -- 172.21.15.112:0/2931798728 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a880059c0 con 0x562533ae54d0 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9a77fff640 1 -- 172.21.15.112:0/2931798728 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f9a8c057ce0 con 0x562533ae54d0 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9aa5bbf740 10 client.25831 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9aa5bbf740 10 client.25831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9aa5bbf740 20 client.25831 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533f32940 0x562533f34d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9aa5bbf740 4 client.25831 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:48:08.078 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.079+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562533e13ee0 con 0x562533f32940 2024-04-25T16:48:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.081+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533f32940 0x562533f34d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.082+0000 7f9a9caa9640 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533f32940 0x562533f34d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ef654a860b194e server_cookie=24162865c5144627 in_seq=0 out_seq=0 2024-04-25T16:48:08.079 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.082+0000 7f9a77fff640 10 client.25831 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.307+0000 7f9a77fff640 1 -- 172.21.15.112:0/2931798728 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562533e13ee0 con 0x562533f32940 2024-04-25T16:48:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.307+0000 7f9a77fff640 10 client.25831 handle_command_reply: tid=0 2024-04-25T16:48:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.307+0000 7f9aa5bbf740 1 client.25831 shutdown 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.307+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 msgr2=0x562533ec1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533ec1f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009900 tx=0x7f9a8c003a90 comp rx=0 tx=0).stop 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 shutdown_connections 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562533f32940 0x562533f34d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562533ae54d0 0x562533ec1f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562533b061b0 0x562533ec24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9a77fff640 0 client.25831 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 --2- 172.21.15.112:0/2931798728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562533a9a1c0 0x562533ec1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 >> 172.21.15.112:0/2931798728 conn(0x562533afa300 msgr2=0x562533ea5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 shutdown_connections 2024-04-25T16:48:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 1 -- 172.21.15.112:0/2931798728 wait complete. 2024-04-25T16:48:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.308+0000 7f9aa5bbf740 20 client.25831 trim_cache size 0 max 16384 2024-04-25T16:48:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:48:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:48:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:48:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9a800afda0 msgr2=0x7f9a800b22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9a800afda0 0x7f9a800b22b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0096c0 tx=0x7f9a8c031400 comp rx=0 tx=0).stop 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 msgr2=0x7f9a98075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98075920 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000eaa0 tx=0x7f9a9000eea0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 shutdown_connections 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f9a800afda0 0x7f9a800b22b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9a9810a040 0x7f9a98075920 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a98109ba0 0x7f9a980753e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 --2- 172.21.15.112:0/1869322890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a981042f0 0x7f9a98074ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 >> 172.21.15.112:0/1869322890 conn(0x7f9a980792a0 msgr2=0x7f9a981022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 shutdown_connections 2024-04-25T16:48:08.312 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:08.309+0000 7f9a9ed0d640 1 -- 172.21.15.112:0/1869322890 wait complete. 2024-04-25T16:48:08.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1030670 10.64 MB/sec execute 374 sec latency 109.141 ms 2024-04-25T16:48:09.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:08 smithi116 ceph-mon[32712]: pgmap v307: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:09.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:08 smithi112 ceph-mon[29122]: pgmap v307: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:09.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:08 smithi133 ceph-mon[32691]: pgmap v307: 97 pgs: 97 active+clean; 95 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 343 op/s 2024-04-25T16:48:09.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1032634 10.63 MB/sec execute 375 sec latency 16.950 ms 2024-04-25T16:48:10.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1034158 10.64 MB/sec execute 376 sec latency 16.392 ms 2024-04-25T16:48:11.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:10 smithi116 ceph-mon[32712]: pgmap v308: 97 pgs: 97 active+clean; 84 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 300 op/s 2024-04-25T16:48:11.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:10 smithi112 ceph-mon[29122]: pgmap v308: 97 pgs: 97 active+clean; 84 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 300 op/s 2024-04-25T16:48:11.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:10 smithi133 ceph-mon[32691]: pgmap v308: 97 pgs: 97 active+clean; 84 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 300 op/s 2024-04-25T16:48:11.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1035757 10.62 MB/sec execute 377 sec latency 68.063 ms 2024-04-25T16:48:12.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1037481 10.63 MB/sec execute 378 sec latency 112.325 ms 2024-04-25T16:48:13.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:13 smithi133 ceph-mon[32691]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 300 op/s 2024-04-25T16:48:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:13 smithi133 ceph-mon[32691]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T16:48:13.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:13 smithi116 ceph-mon[32712]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 300 op/s 2024-04-25T16:48:13.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:13 smithi116 ceph-mon[32712]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T16:48:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:13 smithi112 ceph-mon[29122]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 300 op/s 2024-04-25T16:48:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:13 smithi112 ceph-mon[29122]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T16:48:13.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1038945 10.62 MB/sec execute 379 sec latency 21.726 ms 2024-04-25T16:48:14.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1040984 10.62 MB/sec execute 380 sec latency 26.339 ms 2024-04-25T16:48:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:15 smithi133 ceph-mon[32691]: pgmap v311: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2024-04-25T16:48:15.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:15 smithi116 ceph-mon[32712]: pgmap v311: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2024-04-25T16:48:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:15 smithi112 ceph-mon[29122]: pgmap v311: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2024-04-25T16:48:15.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1042322 10.61 MB/sec execute 381 sec latency 32.592 ms 2024-04-25T16:48:16.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1044488 10.61 MB/sec execute 382 sec latency 26.460 ms 2024-04-25T16:48:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:17 smithi133 ceph-mon[32691]: pgmap v312: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-04-25T16:48:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:17 smithi116 ceph-mon[32712]: pgmap v312: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-04-25T16:48:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:17 smithi112 ceph-mon[29122]: pgmap v312: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-04-25T16:48:17.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1045595 10.60 MB/sec execute 383 sec latency 105.354 ms 2024-04-25T16:48:18.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1047944 10.61 MB/sec execute 384 sec latency 40.987 ms 2024-04-25T16:48:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:19 smithi133 ceph-mon[32691]: pgmap v313: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-04-25T16:48:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:19 smithi116 ceph-mon[32712]: pgmap v313: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-04-25T16:48:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:19 smithi112 ceph-mon[29122]: pgmap v313: 97 pgs: 97 active+clean; 91 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-04-25T16:48:19.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1049072 10.59 MB/sec execute 385 sec latency 20.486 ms 2024-04-25T16:48:20.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1051452 10.60 MB/sec execute 386 sec latency 21.593 ms 2024-04-25T16:48:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 -- 172.21.15.112:0/1436636343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad941031c0 msgr2=0x7fad9410f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 --2- 172.21.15.112:0/1436636343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad941031c0 0x7fad9410f4f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009ca0 tx=0x7fad8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 -- 172.21.15.112:0/1436636343 shutdown_connections 2024-04-25T16:48:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 --2- 172.21.15.112:0/1436636343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad94108f00 0x7fad94102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 --2- 172.21.15.112:0/1436636343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad941031c0 0x7fad9410f4f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 --2- 172.21.15.112:0/1436636343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad94108a60 0x7fad94102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 -- 172.21.15.112:0/1436636343 >> 172.21.15.112:0/1436636343 conn(0x7fad940781b0 msgr2=0x7fad940785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 -- 172.21.15.112:0/1436636343 shutdown_connections 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.410+0000 7fad9abc5640 1 -- 172.21.15.112:0/1436636343 wait complete. 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 Processor -- start 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 -- start start 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad941031c0 0x7fad9419b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 0x7fad9419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad94108f00 0x7fad9419c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112ca0 con 0x7fad94108a60 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112b20 con 0x7fad94108f00 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad9abc5640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad94112e20 con 0x7fad941031c0 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 0x7fad9419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 0x7fad9419bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41176/0 (socket says 172.21.15.112:41176) 2024-04-25T16:48:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad93fff640 1 -- 172.21.15.112:0/635606501 learned_addr learned my addr 172.21.15.112:0/635606501 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad99162640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad94108f00 0x7fad9419c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.411+0000 7fad98961640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad941031c0 0x7fad9419b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad98961640 1 -- 172.21.15.112:0/635606501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad94108f00 msgr2=0x7fad9419c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad98961640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad94108f00 0x7fad9419c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad98961640 1 -- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 msgr2=0x7fad9419bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad98961640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 0x7fad9419bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad98961640 1 -- 172.21.15.112:0/635606501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad941a6910 con 0x7fad941031c0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad93fff640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 0x7fad9419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad98961640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad941031c0 0x7fad9419b600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad94103750 tx=0x7fad8800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad91ffb640 1 -- 172.21.15.112:0/635606501 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8800cc40 con 0x7fad941031c0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad941a6c00 con 0x7fad941031c0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad91ffb640 1 -- 172.21.15.112:0/635606501 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8800cde0 con 0x7fad941031c0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad91ffb640 1 -- 172.21.15.112:0/635606501 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88018760 con 0x7fad941031c0 2024-04-25T16:48:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.412+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad941a71a0 con 0x7fad941031c0 2024-04-25T16:48:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad91ffb640 1 -- 172.21.15.112:0/635606501 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fad88010a90 con 0x7fad941031c0 2024-04-25T16:48:21.419 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 Processor -- start 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 -- start start 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 0x557bdd203980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd203ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd204400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bdd16a460 con 0x557bdce641b0 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557bdd124590 con 0x557bdce3b2f0 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fada1a77740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557bdd166420 con 0x557bdce434d0 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad99162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 0x557bdd203980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad99162640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 0x557bdd203980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41180/0 (socket says 172.21.15.112:41180) 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad99162640 1 -- 172.21.15.112:0/1566927773 learned_addr learned my addr 172.21.15.112:0/1566927773 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:21.420 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad98961640 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd203ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad91ffb640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fad840afc50 0x7fad840b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad93fff640 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd204400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.415+0000 7fad99162640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fad840afc50 0x7fad840b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad99162640 1 -- 172.21.15.112:0/1566927773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 msgr2=0x557bdd204400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad99162640 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd204400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad99162640 1 -- 172.21.15.112:0/1566927773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 msgr2=0x557bdd203ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad99162640 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd203ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad99162640 1 -- 172.21.15.112:0/1566927773 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bdd1310d0 con 0x557bdce641b0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad98961640 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd203ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad91ffb640 1 -- 172.21.15.112:0/635606501 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fad880df350 con 0x7fad941031c0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad99162640 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 0x557bdd203980 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0094e0 tx=0x7fad8c031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a03e03900de6500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad7effd640 1 -- 172.21.15.112:0/1566927773 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004d40 con 0x557bdce641b0 2024-04-25T16:48:21.421 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad7effd640 1 -- 172.21.15.112:0/1566927773 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c036070 con 0x557bdce641b0 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.416+0000 7fad7effd640 1 -- 172.21.15.112:0/1566927773 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c004770 con 0x557bdce641b0 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 -- 172.21.15.112:0/1566927773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 msgr2=0x557bdd203980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 0x557bdd203980 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0094e0 tx=0x7fad8c031e80 comp rx=0 tx=0).stop 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fad99162640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fad840afc50 0x7fad840b2160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad8c04e000 tx=0x7fad8c04c000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 -- 172.21.15.112:0/1566927773 shutdown_connections 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd204400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd203ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 --2- 172.21.15.112:0/1566927773 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce641b0 0x557bdd203980 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 -- 172.21.15.112:0/1566927773 >> 172.21.15.112:0/1566927773 conn(0x557bdce58300 msgr2=0x557bdd2002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 -- 172.21.15.112:0/1566927773 shutdown_connections 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 -- 172.21.15.112:0/1566927773 wait complete. 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 -- start start 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.417+0000 7fada1a77740 1 Processor -- start 2024-04-25T16:48:21.422 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd212b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce641b0 0x557bdd218400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557bdd214f10 con 0x557bdce3b2f0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557bdd0f1b60 con 0x557bdce641b0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557bdd215090 con 0x557bdce434d0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fada1a77740 10 client.0 _ll_register_callbacks cb 0x557bdd1162b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2125c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41188/0 (socket says 172.21.15.112:41188) 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 -- 172.21.15.112:0/1089192810 learned_addr learned my addr 172.21.15.112:0/1089192810 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad93fff640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd212b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad99162640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce641b0 0x557bdd218400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 -- 172.21.15.112:0/1089192810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 msgr2=0x557bdd212b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd212b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.423 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 -- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce641b0 msgr2=0x557bdd218400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce641b0 0x557bdd218400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 -- 172.21.15.112:0/1089192810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bdce7c210 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad93fff640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd212b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad98961640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2125c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fad880a74e0 tx=0x7fad88009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ad1d73a61ef07f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.418+0000 7fad73fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fad73fff640 1 -- 172.21.15.112:0/1089192810 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad880a7320 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fad73fff640 1 -- 172.21.15.112:0/1089192810 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fad880a2ec0 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad8c036070 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fad73fff640 1 -- 172.21.15.112:0/1089192810 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad880a7060 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 10 client.16221 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557bdcf48620 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fad73fff640 1 -- 172.21.15.112:0/1089192810 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad8c036070 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fad73fff640 1 -- 172.21.15.112:0/1089192810 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fad880a1ce0 con 0x557bdce3b2f0 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 10 client.16221 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:21.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 10 client.16221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 20 client.16221 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce22d80 0x557bdd224a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 4 client.16221 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bdd178070 con 0x557bdce22d80 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.419+0000 7fad93fff640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce22d80 0x557bdd224a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.420+0000 7fad73fff640 10 client.16221 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.420+0000 7fad93fff640 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce22d80 0x557bdd224a20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b010e9438c54650 server_cookie=e6770391053df973 in_seq=0 out_seq=0 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.420+0000 7fad73fff640 1 -- 172.21.15.112:0/1089192810 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557bdd178070 con 0x557bdce22d80 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.420+0000 7fad73fff640 10 client.16221 handle_command_reply: tid=0 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 client.16221 shutdown 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 msgr2=0x557bdd2125c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2125c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fad880a74e0 tx=0x7fad88009700 comp rx=0 tx=0).stop 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 shutdown_connections 2024-04-25T16:48:21.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce22d80 0x557bdd224a20 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd212b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fad73fff640 0 client.16221 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce641b0 0x557bdd218400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 --2- 172.21.15.112:0/1089192810 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2125c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 >> 172.21.15.112:0/1089192810 conn(0x557bdce58300 msgr2=0x557bdd204330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 shutdown_connections 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 1 -- 172.21.15.112:0/1089192810 wait complete. 2024-04-25T16:48:21.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.421+0000 7fada1a77740 20 client.16221 trim_cache size 0 max 16384 2024-04-25T16:48:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 Processor -- start 2024-04-25T16:48:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 -- start start 2024-04-25T16:48:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce22d80 0x557bdd212aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.435 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce434d0 0x557bdd20bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad941b0240 con 0x557bdce3b2f0 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad941adb10 con 0x557bdce434d0 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fada1a77740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad941ab980 con 0x557bdce22d80 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fad93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fad93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2182c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41190/0 (socket says 172.21.15.112:41190) 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fad93fff640 1 -- 172.21.15.112:0/1189897291 learned_addr learned my addr 172.21.15.112:0/1189897291 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.436+0000 7fad99162640 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce434d0 0x557bdd20bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad98961640 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce22d80 0x557bdd212aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad93fff640 1 -- 172.21.15.112:0/1189897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce22d80 msgr2=0x557bdd212aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad93fff640 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce22d80 0x557bdd212aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad93fff640 1 -- 172.21.15.112:0/1189897291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce434d0 msgr2=0x557bdd20bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad93fff640 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce434d0 0x557bdd20bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.436 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad93fff640 1 -- 172.21.15.112:0/1189897291 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bdd2116d0 con 0x557bdce3b2f0 2024-04-25T16:48:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad93fff640 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2182c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fad80010d10 tx=0x7fad8000a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a8aa5925f07813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad7e7fc640 1 -- 172.21.15.112:0/1189897291 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8000fd10 con 0x557bdce3b2f0 2024-04-25T16:48:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad7e7fc640 1 -- 172.21.15.112:0/1189897291 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad80024030 con 0x557bdce3b2f0 2024-04-25T16:48:21.437 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.437+0000 7fad7e7fc640 1 -- 172.21.15.112:0/1189897291 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad800047a0 con 0x557bdce3b2f0 2024-04-25T16:48:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 -- 172.21.15.112:0/1189897291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 msgr2=0x557bdd2182c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2182c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fad80010d10 tx=0x7fad8000a2d0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 -- 172.21.15.112:0/1189897291 shutdown_connections 2024-04-25T16:48:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce434d0 0x557bdd20bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.440 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce3b2f0 0x557bdd2182c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 --2- 172.21.15.112:0/1189897291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce22d80 0x557bdd212aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 -- 172.21.15.112:0/1189897291 >> 172.21.15.112:0/1189897291 conn(0x557bdce58300 msgr2=0x557bdd212390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 -- 172.21.15.112:0/1189897291 shutdown_connections 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.438+0000 7fada1a77740 1 -- 172.21.15.112:0/1189897291 wait complete. 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 -- start start 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 Processor -- start 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 0x557bdd229c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd22a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd22a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fad941abfd0 con 0x557bdce22d80 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fad941ad440 con 0x557bdce3b2f0 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad941ac6a0 con 0x557bdce434d0 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fada1a77740 10 client.0 _ll_register_callbacks cb 0x7fad880056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fad98961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 0x557bdd229c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fad93fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd22a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fad98961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 0x557bdd229c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41202/0 (socket says 172.21.15.112:41202) 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fad98961640 1 -- 172.21.15.112:0/2934703270 learned_addr learned my addr 172.21.15.112:0/2934703270 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.439+0000 7fad99162640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd22a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad98961640 1 -- 172.21.15.112:0/2934703270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 msgr2=0x557bdd22a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad98961640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd22a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad98961640 1 -- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 msgr2=0x557bdd22a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad98961640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd22a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad98961640 1 -- 172.21.15.112:0/2934703270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bdd21ca90 con 0x557bdce22d80 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad99162640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd22a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad98961640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 0x557bdd229c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fad880057a0 tx=0x7fad880df2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf5469318064ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad73fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:48:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad73fff640 1 -- 172.21.15.112:0/2934703270 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad880a7320 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad73fff640 1 -- 172.21.15.112:0/2934703270 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fad88010430 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad80024030 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad73fff640 1 -- 172.21.15.112:0/2934703270 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fad880a7550 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 10 client.16227 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557bdd28ce90 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad73fff640 1 -- 172.21.15.112:0/2934703270 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad80024030 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fad73fff640 1 -- 172.21.15.112:0/2934703270 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fad880a1ce0 con 0x557bdce22d80 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 10 client.16227 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 20 client.16227 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce641b0 0x557bdd28f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 4 client.16227 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:48:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.440+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557bdd171ee0 con 0x557bdce641b0 2024-04-25T16:48:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.441+0000 7fad93fff640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce641b0 0x557bdd28f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.441+0000 7fad73fff640 10 client.16227 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.441+0000 7fad93fff640 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce641b0 0x557bdd28f470 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f46134d6624abd2 server_cookie=c004a09c5f89968b in_seq=0 out_seq=0 2024-04-25T16:48:21.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:21 smithi133 ceph-mon[32691]: pgmap v314: 97 pgs: 97 active+clean; 99 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-04-25T16:48:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:21 smithi116 ceph-mon[32712]: pgmap v314: 97 pgs: 97 active+clean; 99 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-04-25T16:48:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:21 smithi112 ceph-mon[29122]: pgmap v314: 97 pgs: 97 active+clean; 99 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fad73fff640 1 -- 172.21.15.112:0/2934703270 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557bdd171ee0 con 0x557bdce641b0 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fad73fff640 10 client.16227 handle_command_reply: tid=0 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 client.16227 shutdown 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 msgr2=0x557bdd229c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 0x557bdd229c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fad880057a0 tx=0x7fad880df2e0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 shutdown_connections 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557bdce641b0 0x557bdd28f470 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557bdce434d0 0x557bdd22a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fad73fff640 0 client.16227 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557bdce3b2f0 0x557bdd22a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 --2- 172.21.15.112:0/2934703270 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557bdce22d80 0x557bdd229c80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 >> 172.21.15.112:0/2934703270 conn(0x557bdce58300 msgr2=0x557bdd203880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 shutdown_connections 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.644+0000 7fada1a77740 1 -- 172.21.15.112:0/2934703270 wait complete. 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fada1a77740 20 client.16227 trim_cache size 0 max 16384 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:48:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:48:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fad840afc50 msgr2=0x7fad840b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fad840afc50 0x7fad840b2160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad8c04e000 tx=0x7fad8c04c000 comp rx=0 tx=0).stop 2024-04-25T16:48:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad941031c0 msgr2=0x7fad9419b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad941031c0 0x7fad9419b600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fad94103750 tx=0x7fad8800ee50 comp rx=0 tx=0).stop 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 shutdown_connections 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fad840afc50 0x7fad840b2160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad94108f00 0x7fad9419c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fad94108a60 0x7fad9419bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 --2- 172.21.15.112:0/635606501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad941031c0 0x7fad9419b600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 >> 172.21.15.112:0/635606501 conn(0x7fad940781b0 msgr2=0x7fad94100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 shutdown_connections 2024-04-25T16:48:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:21.645+0000 7fad9abc5640 1 -- 172.21.15.112:0/635606501 wait complete. 2024-04-25T16:48:21.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1052185 10.58 MB/sec execute 387 sec latency 123.903 ms 2024-04-25T16:48:22.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1053976 10.57 MB/sec execute 388 sec latency 40.709 ms 2024-04-25T16:48:23.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:23 smithi133 ceph-mon[32691]: pgmap v315: 97 pgs: 97 active+clean; 99 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-04-25T16:48:23.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:23 smithi116 ceph-mon[32712]: pgmap v315: 97 pgs: 97 active+clean; 99 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-04-25T16:48:23.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:23 smithi112 ceph-mon[29122]: pgmap v315: 97 pgs: 97 active+clean; 99 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 200 op/s 2024-04-25T16:48:23.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1055479 10.57 MB/sec execute 389 sec latency 16.153 ms 2024-04-25T16:48:24.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1057491 10.57 MB/sec execute 390 sec latency 18.041 ms 2024-04-25T16:48:25.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:25 smithi133 ceph-mon[32691]: pgmap v316: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-04-25T16:48:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:25 smithi116 ceph-mon[32712]: pgmap v316: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-04-25T16:48:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:25 smithi112 ceph-mon[29122]: pgmap v316: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 353 op/s 2024-04-25T16:48:25.889 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1059001 10.57 MB/sec execute 391 sec latency 14.782 ms 2024-04-25T16:48:26.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1061006 10.56 MB/sec execute 392 sec latency 15.787 ms 2024-04-25T16:48:27.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:27 smithi133 ceph-mon[32691]: pgmap v317: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-25T16:48:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:27 smithi116 ceph-mon[32712]: pgmap v317: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-25T16:48:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:27 smithi112 ceph-mon[29122]: pgmap v317: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-25T16:48:27.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1062434 10.56 MB/sec execute 393 sec latency 38.196 ms 2024-04-25T16:48:28.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1063833 10.55 MB/sec execute 394 sec latency 93.696 ms 2024-04-25T16:48:29.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:29 smithi133 ceph-mon[32691]: pgmap v318: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-25T16:48:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:29 smithi116 ceph-mon[32712]: pgmap v318: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-25T16:48:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:29 smithi112 ceph-mon[29122]: pgmap v318: 97 pgs: 97 active+clean; 88 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-25T16:48:29.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1065829 10.56 MB/sec execute 395 sec latency 23.595 ms 2024-04-25T16:48:30.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1067347 10.55 MB/sec execute 396 sec latency 22.821 ms 2024-04-25T16:48:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:31 smithi133 ceph-mon[32691]: pgmap v319: 97 pgs: 97 active+clean; 94 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-04-25T16:48:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:31 smithi116 ceph-mon[32712]: pgmap v319: 97 pgs: 97 active+clean; 94 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-04-25T16:48:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:31 smithi112 ceph-mon[29122]: pgmap v319: 97 pgs: 97 active+clean; 94 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.0 MiB/s wr, 339 op/s 2024-04-25T16:48:31.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1069374 10.55 MB/sec execute 397 sec latency 33.030 ms 2024-04-25T16:48:32.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1070744 10.54 MB/sec execute 398 sec latency 36.407 ms 2024-04-25T16:48:33.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:33 smithi133 ceph-mon[32691]: pgmap v320: 97 pgs: 97 active+clean; 94 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-04-25T16:48:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:33 smithi116 ceph-mon[32712]: pgmap v320: 97 pgs: 97 active+clean; 94 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-04-25T16:48:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:33 smithi112 ceph-mon[29122]: pgmap v320: 97 pgs: 97 active+clean; 94 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-04-25T16:48:33.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1072868 10.55 MB/sec execute 399 sec latency 32.744 ms 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 -- 172.21.15.112:0/2894433444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e241032f0 msgr2=0x7f7e24103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/2894433444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e241032f0 0x7f7e24103700 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009900 tx=0x7f7e1002a410 comp rx=0 tx=0).stop 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 -- 172.21.15.112:0/2894433444 shutdown_connections 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/2894433444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e2410d470 0x7f7e2410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/2894433444 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e241032f0 0x7f7e24103700 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/2894433444 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e24107980 0x7f7e24102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 -- 172.21.15.112:0/2894433444 >> 172.21.15.112:0/2894433444 conn(0x7f7e240778b0 msgr2=0x7f7e24077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 -- 172.21.15.112:0/2894433444 shutdown_connections 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 -- 172.21.15.112:0/2894433444 wait complete. 2024-04-25T16:48:34.752 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.751+0000 7f7e2aff3640 1 Processor -- start 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 -- start start 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 0x7f7e2419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e24107980 0x7f7e2419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 0x7f7e2419c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24112c30 con 0x7f7e2410d470 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24112ab0 con 0x7f7e241032f0 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e2aff3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24112db0 con 0x7f7e24107980 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 0x7f7e2419c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 0x7f7e2419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 0x7f7e2419c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57962/0 (socket says 172.21.15.112:57962) 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 0x7f7e2419b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:54958/0 (socket says 172.21.15.112:54958) 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 -- 172.21.15.112:0/1629690332 learned_addr learned my addr 172.21.15.112:0/1629690332 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e23fff640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e24107980 0x7f7e2419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 -- 172.21.15.112:0/1629690332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e24107980 msgr2=0x7f7e2419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.753 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e24107980 0x7f7e2419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 -- 172.21.15.112:0/1629690332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 msgr2=0x7f7e2419b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 0x7f7e2419b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 -- 172.21.15.112:0/1629690332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e241a68e0 con 0x7f7e2410d470 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 0x7f7e2419b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.752+0000 7f7e29590640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 0x7f7e2419c050 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00eb40 tx=0x7f7e1c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.753+0000 7f7e21ffb640 1 -- 172.21.15.112:0/1629690332 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c004b20 con 0x7f7e2410d470 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.753+0000 7f7e21ffb640 1 -- 172.21.15.112:0/1629690332 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e1c004cc0 con 0x7f7e2410d470 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.753+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e241a6bd0 con 0x7f7e2410d470 2024-04-25T16:48:34.754 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.753+0000 7f7e21ffb640 1 -- 172.21.15.112:0/1629690332 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e1c010720 con 0x7f7e2410d470 2024-04-25T16:48:34.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.753+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a70f0 con 0x7f7e2410d470 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.755+0000 7f7e21ffb640 1 -- 172.21.15.112:0/1629690332 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f7e1c005560 con 0x7f7e2410d470 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.755+0000 7f7e31f7b740 1 Processor -- start 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.755+0000 7f7e31f7b740 1 -- start start 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d751b0 0x564528114af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d4c2f0 0x564528115030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x564528115570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56452807b460 con 0x564527d751b0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564528035590 con 0x564527d544d0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564528077420 con 0x564527d4c2f0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d751b0 0x564528114af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d751b0 0x564528114af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57970/0 (socket says 172.21.15.112:57970) 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d4c2f0 0x564528115030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e23fff640 1 -- 172.21.15.112:0/53616899 learned_addr learned my addr 172.21.15.112:0/53616899 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e23fff640 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x564528115570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e21ffb640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e140afcd0 0x7f7e140b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e29590640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e140afcd0 0x7f7e140b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e28d8f640 1 -- 172.21.15.112:0/53616899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 msgr2=0x564528115570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x564528115570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e28d8f640 1 -- 172.21.15.112:0/53616899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d751b0 msgr2=0x564528114af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d751b0 0x564528114af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e28d8f640 1 -- 172.21.15.112:0/53616899 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645280420d0 con 0x564527d4c2f0 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e23fff640 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x564528115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.756+0000 7f7e21ffb640 1 -- 172.21.15.112:0/1629690332 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f7e1c0df740 con 0x7f7e2410d470 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.757+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d4c2f0 0x564528115030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009fd0 tx=0x7f7e180118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=921f6ba6d12df4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.757+0000 7f7e29590640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e140afcd0 0x7f7e140b21e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0dcef0 tx=0x7f7e1c0ca040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.757+0000 7f7e0effd640 1 -- 172.21.15.112:0/53616899 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18004310 con 0x564527d4c2f0 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.757+0000 7f7e0effd640 1 -- 172.21.15.112:0/53616899 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e180044b0 con 0x564527d4c2f0 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.757+0000 7f7e0effd640 1 -- 172.21.15.112:0/53616899 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18004e30 con 0x564527d4c2f0 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 -- 172.21.15.112:0/53616899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d4c2f0 msgr2=0x564528115030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d4c2f0 0x564528115030 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009fd0 tx=0x7f7e180118d0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 -- 172.21.15.112:0/53616899 shutdown_connections 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d4c2f0 0x564528115030 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x564528115570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/53616899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d751b0 0x564528114af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 -- 172.21.15.112:0/53616899 >> 172.21.15.112:0/53616899 conn(0x564527d69300 msgr2=0x5645281112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 -- 172.21.15.112:0/53616899 shutdown_connections 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 -- 172.21.15.112:0/53616899 wait complete. 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 -- start start 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.758+0000 7f7e31f7b740 1 Processor -- start 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x56452811f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x56452811f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x564528121c20 con 0x564527d544d0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564528002b60 con 0x564527d4c2f0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564528121da0 con 0x564527d751b0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e31f7b740 10 client.0 _ll_register_callbacks cb 0x5645280272b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x56452811f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x56452811f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x56452811f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:54974/0 (socket says 172.21.15.112:54974) 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x56452811f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57982/0 (socket says 172.21.15.112:57982) 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 -- 172.21.15.112:0/1000656405 learned_addr learned my addr 172.21.15.112:0/1000656405 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 -- 172.21.15.112:0/1000656405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 msgr2=0x56452812b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 -- 172.21.15.112:0/1000656405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 msgr2=0x56452811f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x56452811f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 -- 172.21.15.112:0/1000656405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564527d8d210 con 0x564527d4c2f0 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e23fff640 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x56452811f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e100057c0 tx=0x7f7e10031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614ab6e7db330175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.759+0000 7f7e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e03fff640 1 -- 172.21.15.112:0/1000656405 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10035070 con 0x564527d4c2f0 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e180044b0 con 0x564527d4c2f0 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e03fff640 1 -- 172.21.15.112:0/1000656405 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f7e10004460 con 0x564527d4c2f0 2024-04-25T16:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e03fff640 1 -- 172.21.15.112:0/1000656405 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7e10052050 con 0x564527d4c2f0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 10 client.25931 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564527e59620 con 0x564527d4c2f0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e03fff640 1 -- 172.21.15.112:0/1000656405 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e180044b0 con 0x564527d4c2f0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e03fff640 1 -- 172.21.15.112:0/1000656405 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f7e10004b10 con 0x564527d4c2f0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 10 client.25931 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 10 client.25931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 20 client.25931 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d33d80 0x564528131630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 4 client.25931 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564528089070 con 0x564527d33d80 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.760+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d33d80 0x564528131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.761+0000 7f7e03fff640 10 client.25931 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.761+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d33d80 0x564528131630 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63900f6d7fb7481 server_cookie=240656db2847d48e in_seq=0 out_seq=0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e03fff640 1 -- 172.21.15.112:0/1000656405 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564528089070 con 0x564527d33d80 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e03fff640 10 client.25931 handle_command_reply: tid=0 2024-04-25T16:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 client.25931 shutdown 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 msgr2=0x56452811f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x56452811f0e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7e100057c0 tx=0x7f7e10031400 comp rx=0 tx=0).stop 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 shutdown_connections 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d33d80 0x564528131630 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x56452811f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/1000656405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x56452811f0e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e03fff640 0 client.25931 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 >> 172.21.15.112:0/1000656405 conn(0x564527d69300 msgr2=0x564528116810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 shutdown_connections 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 1 -- 172.21.15.112:0/1000656405 wait complete. 2024-04-25T16:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.762+0000 7f7e31f7b740 20 client.25931 trim_cache size 0 max 16384 2024-04-25T16:48:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 Processor -- start 2024-04-25T16:48:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 -- start start 2024-04-25T16:48:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x564528121020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x564528127470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241b2310 con 0x564527d544d0 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241afbe0 con 0x564527d4c2f0 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241ada50 con 0x564527d751b0 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x564528127470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x564528121020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x564528127470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57990/0 (socket says 172.21.15.112:57990) 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 -- 172.21.15.112:0/3360100395 learned_addr learned my addr 172.21.15.112:0/3360100395 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e23fff640 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 -- 172.21.15.112:0/3360100395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 msgr2=0x56452812ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 -- 172.21.15.112:0/3360100395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 msgr2=0x564528121020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x564528121020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.779+0000 7f7e28d8f640 1 -- 172.21.15.112:0/3360100395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564528114880 con 0x564527d544d0 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.780+0000 7f7e23fff640 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x564528121020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.780+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x564528127470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009230 tx=0x7f7e18005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926bc7077a0550f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.780+0000 7f7e0e7fc640 1 -- 172.21.15.112:0/3360100395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e18033070 con 0x564527d544d0 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.780+0000 7f7e0e7fc640 1 -- 172.21.15.112:0/3360100395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e18009b30 con 0x564527d544d0 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.780+0000 7f7e0e7fc640 1 -- 172.21.15.112:0/3360100395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e180037f0 con 0x564527d544d0 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 -- 172.21.15.112:0/3360100395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 msgr2=0x564528127470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x564528127470 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7e18009230 tx=0x7f7e18005f90 comp rx=0 tx=0).stop 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 -- 172.21.15.112:0/3360100395 shutdown_connections 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x56452812ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d544d0 0x564528127470 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/3360100395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d4c2f0 0x564528121020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 -- 172.21.15.112:0/3360100395 >> 172.21.15.112:0/3360100395 conn(0x564527d69300 msgr2=0x564528116730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 -- 172.21.15.112:0/3360100395 shutdown_connections 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 -- 172.21.15.112:0/3360100395 wait complete. 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 -- start start 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.781+0000 7f7e31f7b740 1 Processor -- start 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 0x564528196380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x5645281968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x564528136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241ae0a0 con 0x564527d4c2f0 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241af510 con 0x564527d544d0 2024-04-25T16:48:34.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241ae770 con 0x564527d751b0 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 10 client.0 _ll_register_callbacks cb 0x7f7e100316f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 0x564528196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 0x564528196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57996/0 (socket says 172.21.15.112:57996) 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 -- 172.21.15.112:0/2657626092 learned_addr learned my addr 172.21.15.112:0/2657626092 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x5645281968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 -- 172.21.15.112:0/2657626092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 msgr2=0x564528136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x564528136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 -- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 msgr2=0x5645281968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x5645281968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 -- 172.21.15.112:0/2657626092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564528136e50 con 0x564527d4c2f0 2024-04-25T16:48:34.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x5645281968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e23fff640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 0x564528196380 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006010 tx=0x7f7e10031660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308e883e78ec12ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e03fff640 1 -- 172.21.15.112:0/2657626092 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10035070 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e18009b30 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.782+0000 7f7e03fff640 1 -- 172.21.15.112:0/2657626092 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f7e10003c60 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e03fff640 1 -- 172.21.15.112:0/2657626092 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7e10067050 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 10 client.16245 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56452819da30 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e03fff640 1 -- 172.21.15.112:0/2657626092 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e18009b30 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e03fff640 1 -- 172.21.15.112:0/2657626092 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f7e10062030 con 0x564527d4c2f0 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 10 client.16245 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 20 client.16245 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d091c0 0x5645281a0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 4 client.16245 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:48:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564528082ee0 con 0x564527d091c0 2024-04-25T16:48:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.783+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d091c0 0x5645281a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.784+0000 7f7e03fff640 10 client.16245 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.784+0000 7f7e28d8f640 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d091c0 0x5645281a0040 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5a57bc4103953f server_cookie=369cbbee7d7a19c2 in_seq=0 out_seq=0 2024-04-25T16:48:34.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1074206 10.54 MB/sec execute 400 sec latency 110.286 ms 2024-04-25T16:48:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.960+0000 7f7e03fff640 1 -- 172.21.15.112:0/2657626092 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564528082ee0 con 0x564527d091c0 2024-04-25T16:48:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.960+0000 7f7e03fff640 10 client.16245 handle_command_reply: tid=0 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.960+0000 7f7e31f7b740 1 client.16245 shutdown 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.960+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 msgr2=0x564528196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.960+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 0x564528196380 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7e10006010 tx=0x7f7e10031660 comp rx=0 tx=0).stop 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.960+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 shutdown_connections 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564527d091c0 0x5645281a0040 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564527d751b0 0x564528136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564527d544d0 0x5645281968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 --2- 172.21.15.112:0/2657626092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564527d4c2f0 0x564528196380 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 >> 172.21.15.112:0/2657626092 conn(0x564527d69300 msgr2=0x564528120c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e03fff640 0 client.16245 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 shutdown_connections 2024-04-25T16:48:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 1 -- 172.21.15.112:0/2657626092 wait complete. 2024-04-25T16:48:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.961+0000 7f7e31f7b740 20 client.16245 trim_cache size 0 max 16384 2024-04-25T16:48:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:48:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:48:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:48:34.962 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:48:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e140afcd0 msgr2=0x7f7e140b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e140afcd0 0x7f7e140b21e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c0dcef0 tx=0x7f7e1c0ca040 comp rx=0 tx=0).stop 2024-04-25T16:48:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 msgr2=0x7f7e2419c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 0x7f7e2419c050 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00eb40 tx=0x7f7e1c00c2d0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 shutdown_connections 2024-04-25T16:48:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e140afcd0 0x7f7e140b21e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e24107980 0x7f7e2419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e2410d470 0x7f7e2419c050 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 --2- 172.21.15.112:0/1629690332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e241032f0 0x7f7e2419b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 >> 172.21.15.112:0/1629690332 conn(0x7f7e240778b0 msgr2=0x7f7e24100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 shutdown_connections 2024-04-25T16:48:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:34.962+0000 7f7e2aff3640 1 -- 172.21.15.112:0/1629690332 wait complete. 2024-04-25T16:48:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:35 smithi133 ceph-mon[32691]: pgmap v321: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-04-25T16:48:35.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:35 smithi116 ceph-mon[32712]: pgmap v321: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-04-25T16:48:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:35 smithi112 ceph-mon[29122]: pgmap v321: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 335 op/s 2024-04-25T16:48:35.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1076335 10.54 MB/sec execute 401 sec latency 140.839 ms 2024-04-25T16:48:36.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1077737 10.53 MB/sec execute 402 sec latency 20.257 ms 2024-04-25T16:48:37.396 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:37 smithi112 ceph-mon[29122]: pgmap v322: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-25T16:48:37.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:37 smithi133 ceph-mon[32691]: pgmap v322: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-25T16:48:37.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:37 smithi116 ceph-mon[32712]: pgmap v322: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-25T16:48:37.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1079882 10.54 MB/sec execute 403 sec latency 36.638 ms 2024-04-25T16:48:38.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1081093 10.52 MB/sec execute 404 sec latency 34.286 ms 2024-04-25T16:48:39.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:39 smithi133 ceph-mon[32691]: pgmap v323: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-25T16:48:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:39 smithi116 ceph-mon[32712]: pgmap v323: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-25T16:48:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:39 smithi112 ceph-mon[29122]: pgmap v323: 97 pgs: 97 active+clean; 100 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-04-25T16:48:39.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1083426 10.53 MB/sec execute 405 sec latency 20.204 ms 2024-04-25T16:48:40.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1084634 10.51 MB/sec execute 406 sec latency 28.149 ms 2024-04-25T16:48:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:41 smithi133 ceph-mon[32691]: pgmap v324: 97 pgs: 97 active+clean; 90 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 334 op/s 2024-04-25T16:48:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:41 smithi116 ceph-mon[32712]: pgmap v324: 97 pgs: 97 active+clean; 90 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 334 op/s 2024-04-25T16:48:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:41 smithi112 ceph-mon[29122]: pgmap v324: 97 pgs: 97 active+clean; 90 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 334 op/s 2024-04-25T16:48:41.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1086975 10.53 MB/sec execute 407 sec latency 24.334 ms 2024-04-25T16:48:42.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:42 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:48:42.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:42 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:48:42.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:42 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:48:42.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1087950 10.51 MB/sec execute 408 sec latency 99.637 ms 2024-04-25T16:48:43.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:43 smithi133 ceph-mon[32691]: pgmap v325: 97 pgs: 97 active+clean; 90 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 253 op/s 2024-04-25T16:48:43.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:43 smithi116 ceph-mon[32712]: pgmap v325: 97 pgs: 97 active+clean; 90 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 253 op/s 2024-04-25T16:48:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:43 smithi112 ceph-mon[29122]: pgmap v325: 97 pgs: 97 active+clean; 90 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 253 op/s 2024-04-25T16:48:43.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1090169 10.51 MB/sec execute 409 sec latency 36.945 ms 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: pgmap v326: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 332 op/s 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: Unable to set osd_memory_target on smithi112 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: Unable to set osd_memory_target on smithi133 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:44 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: pgmap v326: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 332 op/s 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: Unable to set osd_memory_target on smithi112 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-25T16:48:44.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: Unable to set osd_memory_target on smithi133 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T16:48:44.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.846 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:44 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1091199 10.50 MB/sec execute 410 sec latency 19.051 ms 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: pgmap v326: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 332 op/s 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi112 to 176.2M 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: Unable to set osd_memory_target on smithi112 to 184763084: error parsing value: Value '184763084' is below minimum 939524096 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi133 to 176.2M 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: Unable to set osd_memory_target on smithi133 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:44.950 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:44 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: Adjusting osd_memory_target on smithi116 to 176.2M 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: Unable to set osd_memory_target on smithi116 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:48:45.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: Adjusting osd_memory_target on smithi116 to 176.2M 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: Unable to set osd_memory_target on smithi116 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:48:45.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:45.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1093482 10.50 MB/sec execute 411 sec latency 19.708 ms 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: Adjusting osd_memory_target on smithi116 to 176.2M 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: Unable to set osd_memory_target on smithi116 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:48:45.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:48:46.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:46 smithi116 ceph-mon[32712]: pgmap v327: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-04-25T16:48:46.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:46 smithi112 ceph-mon[29122]: pgmap v327: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-04-25T16:48:46.890 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1094770 10.50 MB/sec execute 412 sec latency 18.327 ms 2024-04-25T16:48:46.948 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:46 smithi133 ceph-mon[32691]: pgmap v327: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-04-25T16:48:47.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1096847 10.50 MB/sec execute 413 sec latency 59.515 ms 2024-04-25T16:48:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.093+0000 7f61bae97640 1 -- 172.21.15.112:0/350765861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 msgr2=0x7f61ac095d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.093+0000 7f61bae97640 1 --2- 172.21.15.112:0/350765861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac095d50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009900 tx=0x7f61a402a4b0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 -- 172.21.15.112:0/350765861 shutdown_connections 2024-04-25T16:48:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 --2- 172.21.15.112:0/350765861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61ac09ad30 0x7f61ac09b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 --2- 172.21.15.112:0/350765861 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac09a390 0x7f61ac09a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 --2- 172.21.15.112:0/350765861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac095d50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 -- 172.21.15.112:0/350765861 >> 172.21.15.112:0/350765861 conn(0x7f61ac00aff0 msgr2=0x7f61ac00b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 -- 172.21.15.112:0/350765861 shutdown_connections 2024-04-25T16:48:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.094+0000 7f61bae97640 1 -- 172.21.15.112:0/350765861 wait complete. 2024-04-25T16:48:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.095+0000 7f61bae97640 1 Processor -- start 2024-04-25T16:48:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.095+0000 7f61bae97640 1 -- start start 2024-04-25T16:48:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61bae97640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac139860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61bae97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac09a390 0x7f61ac139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61bae97640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61ac09ad30 0x7f61ac133ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61bae97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac0ad240 con 0x7f61ac09a390 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61bae97640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac0ad0c0 con 0x7f61ac09ad30 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61bae97640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac0ad3c0 con 0x7f61ac095940 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:37988/0 (socket says 172.21.15.112:37988) 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 -- 172.21.15.112:0/4090339155 learned_addr learned my addr 172.21.15.112:0/4090339155 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61ba696640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61ac09ad30 0x7f61ac133ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9694640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac09a390 0x7f61ac139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 -- 172.21.15.112:0/4090339155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61ac09ad30 msgr2=0x7f61ac133ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61ac09ad30 0x7f61ac133ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 -- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac09a390 msgr2=0x7f61ac139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac09a390 0x7f61ac139da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.096+0000 7f61b9e95640 1 -- 172.21.15.112:0/4090339155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac134160 con 0x7f61ac095940 2024-04-25T16:48:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.097+0000 7f61b9e95640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac139860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61a4005ab0 tx=0x7f61a4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.097+0000 7f61aaffd640 1 -- 172.21.15.112:0/4090339155 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a403f070 con 0x7f61ac095940 2024-04-25T16:48:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.097+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac1343f0 con 0x7f61ac095940 2024-04-25T16:48:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.097+0000 7f61aaffd640 1 -- 172.21.15.112:0/4090339155 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61a4031b60 con 0x7f61ac095940 2024-04-25T16:48:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.097+0000 7f61aaffd640 1 -- 172.21.15.112:0/4090339155 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4035030 con 0x7f61ac095940 2024-04-25T16:48:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.097+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac1406e0 con 0x7f61ac095940 2024-04-25T16:48:48.097 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 Processor -- start 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 -- start start 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e9b1b0 0x56095423ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e722f0 0x56095423b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x56095423b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5609541a1460 con 0x560953e7a4d0 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56095415b590 con 0x560953e722f0 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61c2fab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56095419d420 con 0x560953e9b1b0 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61b9694640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x56095423b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61ba696640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e9b1b0 0x56095423ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61b9694640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x56095423b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53128/0 (socket says 172.21.15.112:53128) 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61aaffd640 1 -- 172.21.15.112:0/4090339155 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f61a4042070 con 0x7f61ac095940 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.100+0000 7f61b9694640 1 -- 172.21.15.112:0/3477905625 learned_addr learned my addr 172.21.15.112:0/3477905625 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9e95640 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e722f0 0x56095423b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9694640 1 -- 172.21.15.112:0/3477905625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e9b1b0 msgr2=0x56095423ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9694640 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e9b1b0 0x56095423ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9694640 1 -- 172.21.15.112:0/3477905625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e722f0 msgr2=0x56095423b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9694640 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e722f0 0x56095423b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9694640 1 -- 172.21.15.112:0/3477905625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609541680d0 con 0x560953e7a4d0 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9e95640 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e722f0 0x56095423b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61b9694640 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x56095423b710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009c60 tx=0x7f61b0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1458871c2ebafa43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f619ffff640 1 -- 172.21.15.112:0/3477905625 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004230 con 0x560953e7a4d0 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f619ffff640 1 -- 172.21.15.112:0/3477905625 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b00043d0 con 0x560953e7a4d0 2024-04-25T16:48:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f619ffff640 1 -- 172.21.15.112:0/3477905625 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004230 con 0x560953e7a4d0 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.101+0000 7f61aaffd640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f61a00afce0 0x7f61a00b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61aaffd640 1 -- 172.21.15.112:0/4090339155 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f61a40ff7b0 con 0x7f61ac095940 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 -- 172.21.15.112:0/3477905625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 msgr2=0x56095423b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x56095423b710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009c60 tx=0x7f61b0011810 comp rx=0 tx=0).stop 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61ba696640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f61a00afce0 0x7f61a00b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 -- 172.21.15.112:0/3477905625 shutdown_connections 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x56095423b710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e722f0 0x56095423b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 --2- 172.21.15.112:0/3477905625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e9b1b0 0x56095423ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 -- 172.21.15.112:0/3477905625 >> 172.21.15.112:0/3477905625 conn(0x560953e8f300 msgr2=0x560954237440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 -- 172.21.15.112:0/3477905625 shutdown_connections 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.102+0000 7f61c2fab740 1 -- 172.21.15.112:0/3477905625 wait complete. 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61ba696640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f61a00afce0 0x7f61a00b21f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61b40514f0 tx=0x7f61b4077040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 -- start start 2024-04-25T16:48:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 Processor -- start 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x5609542452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x5609542457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e9b1b0 0x560954251270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x560954247dc0 con 0x560953e722f0 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560954128b60 con 0x560953e9b1b0 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560954247f40 con 0x560953e7a4d0 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61c2fab740 10 client.0 _ll_register_callbacks cb 0x56095414d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61b9e95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x5609542452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.103+0000 7f61b9e95640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x5609542452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53134/0 (socket says 172.21.15.112:53134) 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 -- 172.21.15.112:0/2309033358 learned_addr learned my addr 172.21.15.112:0/2309033358 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9694640 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x5609542457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 -- 172.21.15.112:0/2309033358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 msgr2=0x5609542457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x5609542457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 -- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e9b1b0 msgr2=0x560954251270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e9b1b0 0x560954251270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 -- 172.21.15.112:0/2309033358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560953eb3210 con 0x560953e722f0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61b9e95640 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x5609542452b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f61a40c7650 tx=0x7f61a4031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d558943e27ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f619d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f619d7fa640 1 -- 172.21.15.112:0/2309033358 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4049070 con 0x560953e722f0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b00043d0 con 0x560953e722f0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.104+0000 7f619d7fa640 1 -- 172.21.15.112:0/2309033358 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f61a40fd0d0 con 0x560953e722f0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.105+0000 7f619d7fa640 1 -- 172.21.15.112:0/2309033358 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61a40c7460 con 0x560953e722f0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.105+0000 7f61c2fab740 10 client.16257 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.105+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560953f7f620 con 0x560953e722f0 2024-04-25T16:48:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.105+0000 7f619d7fa640 1 -- 172.21.15.112:0/2309033358 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b00043d0 con 0x560953e722f0 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f619d7fa640 1 -- 172.21.15.112:0/2309033358 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f61a40c3be0 con 0x560953e722f0 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f61c2fab740 10 client.16257 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f61c2fab740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f61c2fab740 20 client.16257 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e59d80 0x560954257800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f61c2fab740 4 client.16257 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:48:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.106+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609541af070 con 0x560953e59d80 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.107+0000 7f61b9694640 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e59d80 0x560954257800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.108+0000 7f619d7fa640 10 client.16257 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.108+0000 7f61b9694640 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e59d80 0x560954257800 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7556f6826b49c9d4 server_cookie=2a98a2c9eb6d1b62 in_seq=0 out_seq=0 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f619d7fa640 1 -- 172.21.15.112:0/2309033358 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609541af070 con 0x560953e59d80 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f619d7fa640 10 client.16257 handle_command_reply: tid=0 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 client.16257 shutdown 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 msgr2=0x5609542452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x5609542452b0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f61a40c7650 tx=0x7f61a4031f40 comp rx=0 tx=0).stop 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 shutdown_connections 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e59d80 0x560954257800 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f619d7fa640 0 client.16257 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x5609542457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e9b1b0 0x560954251270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2309033358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x5609542452b0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 >> 172.21.15.112:0/2309033358 conn(0x560953e8f300 msgr2=0x56095423ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 shutdown_connections 2024-04-25T16:48:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.109+0000 7f61c2fab740 1 -- 172.21.15.112:0/2309033358 wait complete. 2024-04-25T16:48:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.110+0000 7f61c2fab740 20 client.16257 trim_cache size 0 max 16384 2024-04-25T16:48:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.129+0000 7f61c2fab740 1 Processor -- start 2024-04-25T16:48:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.129+0000 7f61c2fab740 1 -- start start 2024-04-25T16:48:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x560954237530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x560954250e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x56095425d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61c2fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac14b800 con 0x560953e722f0 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61c2fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1490d0 con 0x560953e59d80 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61c2fab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac146f40 con 0x560953e7a4d0 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x560954250e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x560954250e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53148/0 (socket says 172.21.15.112:53148) 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 -- 172.21.15.112:0/2403057744 learned_addr learned my addr 172.21.15.112:0/2403057744 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9e95640 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x560954237530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61ba696640 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x56095425d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 -- 172.21.15.112:0/2403057744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 msgr2=0x56095425d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x56095425d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 -- 172.21.15.112:0/2403057744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 msgr2=0x560954237530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x560954237530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 -- 172.21.15.112:0/2403057744 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560954245fc0 con 0x560953e722f0 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61ba696640 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x56095425d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f61b9694640 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x560954250e50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f61b00048b0 tx=0x7f61b00048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ad287749c3425e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f619f7fe640 1 -- 172.21.15.112:0/2403057744 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b001e040 con 0x560953e722f0 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.130+0000 7f619f7fe640 1 -- 172.21.15.112:0/2403057744 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b001f040 con 0x560953e722f0 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f619f7fe640 1 -- 172.21.15.112:0/2403057744 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0004230 con 0x560953e722f0 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 -- 172.21.15.112:0/2403057744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 msgr2=0x560954250e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x560954250e50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f61b00048b0 tx=0x7f61b00048e0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 -- 172.21.15.112:0/2403057744 shutdown_connections 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e7a4d0 0x56095425d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e722f0 0x560954250e50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 --2- 172.21.15.112:0/2403057744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x560954237530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 -- 172.21.15.112:0/2403057744 >> 172.21.15.112:0/2403057744 conn(0x560953e8f300 msgr2=0x560954236e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 -- 172.21.15.112:0/2403057744 shutdown_connections 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.131+0000 7f61c2fab740 1 -- 172.21.15.112:0/2403057744 wait complete. 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 -- start start 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 Processor -- start 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x56095424d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e722f0 0x56095424d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x5609542488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac147590 con 0x560953e7a4d0 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac148a00 con 0x560953e59d80 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac147c60 con 0x560953e722f0 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.132+0000 7f61c2fab740 10 client.0 _ll_register_callbacks cb 0x7f61a4005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:48:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x5609542488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x5609542488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:53150/0 (socket says 172.21.15.112:53150) 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 -- 172.21.15.112:0/1854453277 learned_addr learned my addr 172.21.15.112:0/1854453277 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61b9e95640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x56095424d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61b9694640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e722f0 0x56095424d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 -- 172.21.15.112:0/1854453277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e722f0 msgr2=0x56095424d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e722f0 0x56095424d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 -- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 msgr2=0x56095424d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x56095424d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 -- 172.21.15.112:0/1854453277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560954249030 con 0x560953e7a4d0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61b9e95640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x56095424d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61ba696640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x5609542488a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f61b4050a10 tx=0x7f61b4068b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a5050349475610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f619d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f619d7fa640 1 -- 172.21.15.112:0/1854453277 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b4093070 con 0x560953e7a4d0 2024-04-25T16:48:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f619d7fa640 1 -- 172.21.15.112:0/1854453277 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f61b4082d30 con 0x560953e7a4d0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61b001f040 con 0x560953e7a4d0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f619d7fa640 1 -- 172.21.15.112:0/1854453277 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f61b408f420 con 0x560953e7a4d0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61c2fab740 10 client.16263 fetch_fsmap learned FSMap version 34 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.133+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609542bf740 con 0x560953e7a4d0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f619d7fa640 1 -- 172.21.15.112:0/1854453277 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b001f040 con 0x560953e7a4d0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f619d7fa640 1 -- 172.21.15.112:0/1854453277 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f61b4097540 con 0x560953e7a4d0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61c2fab740 10 client.16263 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61c2fab740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61c2fab740 20 client.16263 populate_metadata read hostname 'smithi112' 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61c2fab740 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e9b1b0 0x5609542c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61c2fab740 4 client.16263 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609541a8ee0 con 0x560953e9b1b0 2024-04-25T16:48:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.134+0000 7f61b9e95640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e9b1b0 0x5609542c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:48:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.135+0000 7f619d7fa640 10 client.16263 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.135+0000 7f61b9e95640 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e9b1b0 0x5609542c1ea0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a824a75b5c3eaaf7 server_cookie=696e6a0e7a31c815 in_seq=0 out_seq=0 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f619d7fa640 1 -- 172.21.15.112:0/1854453277 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609541a8ee0 con 0x560953e9b1b0 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f619d7fa640 10 client.16263 handle_command_reply: tid=0 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 client.16263 shutdown 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 msgr2=0x5609542488a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x5609542488a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f61b4050a10 tx=0x7f61b4068b80 comp rx=0 tx=0).stop 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 shutdown_connections 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x560953e9b1b0 0x5609542c1ea0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f619d7fa640 0 client.16263 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560953e722f0 0x56095424d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x560953e7a4d0 0x5609542488a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.307+0000 7f61c2fab740 1 --2- 172.21.15.112:0/1854453277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560953e59d80 0x56095424d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.308+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 >> 172.21.15.112:0/1854453277 conn(0x560953e8f300 msgr2=0x560954246670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.308+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 shutdown_connections 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.308+0000 7f61c2fab740 1 -- 172.21.15.112:0/1854453277 wait complete. 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.308+0000 7f61c2fab740 20 client.16263 trim_cache size 0 max 16384 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:48:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:48:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f61a00afce0 msgr2=0x7f61a00b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f61a00afce0 0x7f61a00b21f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f61b40514f0 tx=0x7f61b4077040 comp rx=0 tx=0).stop 2024-04-25T16:48:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 msgr2=0x7f61ac139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:48:48.308 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac139860 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f61a4005ab0 tx=0x7f61a4003c10 comp rx=0 tx=0).stop 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 shutdown_connections 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f61a00afce0 0x7f61a00b21f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61ac09ad30 0x7f61ac133ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61ac09a390 0x7f61ac139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 --2- 172.21.15.112:0/4090339155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61ac095940 0x7f61ac139860 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.309+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 >> 172.21.15.112:0/4090339155 conn(0x7f61ac00aff0 msgr2=0x7f61ac098360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.310+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 shutdown_connections 2024-04-25T16:48:48.309 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:48:48.310+0000 7f61bae97640 1 -- 172.21.15.112:0/4090339155 wait complete. 2024-04-25T16:48:48.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1098044 10.49 MB/sec execute 414 sec latency 75.501 ms 2024-04-25T16:48:49.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:49 smithi133 ceph-mon[32691]: pgmap v328: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-04-25T16:48:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:49 smithi116 ceph-mon[32712]: pgmap v328: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-04-25T16:48:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:49 smithi112 ceph-mon[29122]: pgmap v328: 97 pgs: 97 active+clean; 97 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-04-25T16:48:49.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1100056 10.48 MB/sec execute 415 sec latency 19.931 ms 2024-04-25T16:48:50.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1101763 10.49 MB/sec execute 416 sec latency 19.665 ms 2024-04-25T16:48:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:51 smithi133 ceph-mon[32691]: pgmap v329: 97 pgs: 97 active+clean; 86 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-04-25T16:48:51.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:51 smithi116 ceph-mon[32712]: pgmap v329: 97 pgs: 97 active+clean; 86 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-04-25T16:48:51.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:51 smithi112 ceph-mon[29122]: pgmap v329: 97 pgs: 97 active+clean; 86 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.6 MiB/s wr, 409 op/s 2024-04-25T16:48:51.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1104114 10.49 MB/sec execute 417 sec latency 21.546 ms 2024-04-25T16:48:52.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1105257 10.48 MB/sec execute 418 sec latency 38.661 ms 2024-04-25T16:48:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:53 smithi133 ceph-mon[32691]: pgmap v330: 97 pgs: 97 active+clean; 86 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-04-25T16:48:53.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:53 smithi116 ceph-mon[32712]: pgmap v330: 97 pgs: 97 active+clean; 86 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-04-25T16:48:53.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:53 smithi112 ceph-mon[29122]: pgmap v330: 97 pgs: 97 active+clean; 86 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-04-25T16:48:53.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1107303 10.48 MB/sec execute 419 sec latency 32.199 ms 2024-04-25T16:48:54.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1108779 10.48 MB/sec execute 420 sec latency 15.501 ms 2024-04-25T16:48:55.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:55 smithi133 ceph-mon[32691]: pgmap v331: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 324 op/s 2024-04-25T16:48:55.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:55 smithi116 ceph-mon[32712]: pgmap v331: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 324 op/s 2024-04-25T16:48:55.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:55 smithi112 ceph-mon[29122]: pgmap v331: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 324 op/s 2024-04-25T16:48:55.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1110954 10.48 MB/sec execute 421 sec latency 22.179 ms 2024-04-25T16:48:56.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1112463 10.47 MB/sec execute 422 sec latency 18.354 ms 2024-04-25T16:48:57.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:57 smithi133 ceph-mon[32691]: pgmap v332: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-25T16:48:57.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:57 smithi116 ceph-mon[32712]: pgmap v332: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-25T16:48:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:57 smithi112 ceph-mon[29122]: pgmap v332: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-25T16:48:57.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1114702 10.48 MB/sec execute 423 sec latency 18.507 ms 2024-04-25T16:48:58.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1115744 10.47 MB/sec execute 424 sec latency 93.544 ms 2024-04-25T16:48:59.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:48:59 smithi133 ceph-mon[32691]: pgmap v333: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-25T16:48:59.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:48:59 smithi116 ceph-mon[32712]: pgmap v333: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-25T16:48:59.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:48:59 smithi112 ceph-mon[29122]: pgmap v333: 97 pgs: 97 active+clean; 93 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-04-25T16:48:59.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1117717 10.46 MB/sec execute 425 sec latency 15.806 ms 2024-04-25T16:49:00.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1119062 10.46 MB/sec execute 426 sec latency 28.234 ms 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 -- 172.21.15.112:0/2888797006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f01031e0 msgr2=0x7fb1f010f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 --2- 172.21.15.112:0/2888797006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f01031e0 0x7fb1f010f5b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009ca0 tx=0x7fb1e802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 -- 172.21.15.112:0/2888797006 shutdown_connections 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 --2- 172.21.15.112:0/2888797006 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f01031e0 0x7fb1f010f5b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 --2- 172.21.15.112:0/2888797006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1f0108f30 0x7fb1f0102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 --2- 172.21.15.112:0/2888797006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f0108a90 0x7fb1f0102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 -- 172.21.15.112:0/2888797006 >> 172.21.15.112:0/2888797006 conn(0x7fb1f00fe260 msgr2=0x7fb1f0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 -- 172.21.15.112:0/2888797006 shutdown_connections 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.414+0000 7fb1f8772640 1 -- 172.21.15.112:0/2888797006 wait complete. 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 Processor -- start 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 -- start start 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f01031e0 0x7fb1f00fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 0x7fb1f00fec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1f0108f30 0x7fb1f00ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112ca0 con 0x7fb1f0108a90 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112b20 con 0x7fb1f0108f30 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f8772640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f0112e20 con 0x7fb1f01031e0 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 0x7fb1f00fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 0x7fb1f00fec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56690/0 (socket says 172.21.15.112:56690) 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/301930849 learned_addr learned my addr 172.21.15.112:0/301930849 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.415+0000 7fb1f650e640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f01031e0 0x7fb1f00fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1f0108f30 0x7fb1f00ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/301930849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f01031e0 msgr2=0x7fb1f00fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f01031e0 0x7fb1f00fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/301930849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1f0108f30 msgr2=0x7fb1f00ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1f0108f30 0x7fb1f00ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/301930849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f01a8a50 con 0x7fb1f0108a90 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 0x7fb1f00fec80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000e9f0 tx=0x7fb1e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1e77fe640 1 -- 172.21.15.112:0/301930849 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000ccc0 con 0x7fb1f0108a90 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1e77fe640 1 -- 172.21.15.112:0/301930849 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00035f0 con 0x7fb1f0108a90 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f01a8d40 con 0x7fb1f0108a90 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1e77fe640 1 -- 172.21.15.112:0/301930849 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e000ce60 con 0x7fb1f0108a90 2024-04-25T16:49:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.416+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f01a9220 con 0x7fb1f0108a90 2024-04-25T16:49:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1e77fe640 1 -- 172.21.15.112:0/301930849 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fb1e0003aa0 con 0x7fb1f0108a90 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 Processor -- start 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 -- start start 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 0x557e92327af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e92328030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f674d0 0x557e92328570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e9228e460 con 0x557e91f881b0 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557e92248590 con 0x557e91f674d0 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557e9228a420 con 0x557e91f5f2f0 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1f6d0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 0x557e92327af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f674d0 0x557e92328570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1f6d0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 0x557e92327af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56696/0 (socket says 172.21.15.112:56696) 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1f6d0f640 1 -- 172.21.15.112:0/1238370942 learned_addr learned my addr 172.21.15.112:0/1238370942 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.419+0000 7fb1f650e640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e92328030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1e77fe640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb1dc0afcd0 0x7fb1dc0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/1238370942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 msgr2=0x557e92328030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e92328030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/1238370942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 msgr2=0x557e92327af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 0x557e92327af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/1238370942 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e922550d0 con 0x557e91f674d0 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 0x557e92327af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f650e640 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e92328030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb1dc0afcd0 0x7fb1dc0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1e77fe640 1 -- 172.21.15.112:0/301930849 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fb1e00df2b0 con 0x7fb1f0108a90 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f674d0 0x557e92328570 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00dc9b0 tx=0x7fb1e0021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564ad36fbbd4cd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1e4ff9640 1 -- 172.21.15.112:0/1238370942 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00a71d0 con 0x557e91f674d0 2024-04-25T16:49:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb1dc0afcd0 0x7fb1dc0b21e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557e92324c60 tx=0x7fb1e80095c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1e4ff9640 1 -- 172.21.15.112:0/1238370942 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00a6f10 con 0x557e91f674d0 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.420+0000 7fb1e4ff9640 1 -- 172.21.15.112:0/1238370942 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00a6b70 con 0x557e91f674d0 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 -- 172.21.15.112:0/1238370942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f674d0 msgr2=0x557e92328570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f674d0 0x557e92328570 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00dc9b0 tx=0x7fb1e0021000 comp rx=0 tx=0).stop 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 -- 172.21.15.112:0/1238370942 shutdown_connections 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e92328030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f674d0 0x557e92328570 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/1238370942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f881b0 0x557e92327af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 -- 172.21.15.112:0/1238370942 >> 172.21.15.112:0/1238370942 conn(0x557e91f7c300 msgr2=0x557e923242a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 -- 172.21.15.112:0/1238370942 shutdown_connections 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.421+0000 7fb1ff61e740 1 -- 172.21.15.112:0/1238370942 wait complete. 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 -- start start 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 Processor -- start 2024-04-25T16:49:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 0x557e92333b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f674d0 0x557e92334040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92331ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x557e92334bc0 con 0x557e91f5f2f0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557e92215b60 con 0x557e91f881b0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557e92334d40 con 0x557e91f674d0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1ff61e740 10 client.0 _ll_register_callbacks cb 0x557e9223a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 0x557e92333b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 0x557e92333b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56698/0 (socket says 172.21.15.112:56698) 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 -- 172.21.15.112:0/134312071 learned_addr learned my addr 172.21.15.112:0/134312071 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92331ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f674d0 0x557e92334040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 -- 172.21.15.112:0/134312071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f674d0 msgr2=0x557e92334040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f674d0 0x557e92334040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 -- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 msgr2=0x557e92331ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92331ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.422+0000 7fb1f650e640 1 -- 172.21.15.112:0/134312071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e91fa0210 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1f650e640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 0x557e92333b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00b8f0 tx=0x7fb1ec00bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40093ba63c327844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/134312071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1ec003d60 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/134312071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fb1ec011440 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e00a6f10 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/134312071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb1ec005020 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 10 client.16275 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e9206c620 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/134312071 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00a6f10 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/134312071 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fb1ec011640 con 0x557e91f5f2f0 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 10 client.16275 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 10 client.16275 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 20 client.16275 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f46d80 0x557e92344780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 4 client.16275 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.423+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9229c070 con 0x557e91f46d80 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.424+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f46d80 0x557e92344780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.424+0000 7fb1d9ffb640 10 client.16275 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.424+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f46d80 0x557e92344780 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab3b7bc26f890a8e server_cookie=4fac6e2e443afd44 in_seq=0 out_seq=0 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/134312071 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e9229c070 con 0x557e91f46d80 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1d9ffb640 10 client.16275 handle_command_reply: tid=0 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 client.16275 shutdown 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 msgr2=0x557e92333b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 0x557e92333b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec00b8f0 tx=0x7fb1ec00bdb0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 shutdown_connections 2024-04-25T16:49:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f46d80 0x557e92344780 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f674d0 0x557e92334040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92331ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1d9ffb640 0 client.16275 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/134312071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f5f2f0 0x557e92333b00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.426+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 >> 172.21.15.112:0/134312071 conn(0x557e91f7c300 msgr2=0x557e92329810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.427+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 shutdown_connections 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.427+0000 7fb1ff61e740 1 -- 172.21.15.112:0/134312071 wait complete. 2024-04-25T16:49:01.431 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.427+0000 7fb1ff61e740 20 client.16275 trim_cache size 0 max 16384 2024-04-25T16:49:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 Processor -- start 2024-04-25T16:49:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 -- start start 2024-04-25T16:49:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f5f2f0 0x557e92333e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.444 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e9232b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f881b0 0x557e9232bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b4680 con 0x557e91f674d0 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b1f50 con 0x557e91f5f2f0 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01afdc0 con 0x557e91f881b0 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e9232b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e9232b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56704/0 (socket says 172.21.15.112:56704) 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2626097616 learned_addr learned my addr 172.21.15.112:0/2626097616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f650e640 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f5f2f0 0x557e92333e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f881b0 0x557e9232bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2626097616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f881b0 msgr2=0x557e9232bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f881b0 0x557e9232bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2626097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f5f2f0 msgr2=0x557e92333e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f5f2f0 0x557e92333e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2626097616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e92328e80 con 0x557e91f674d0 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e9232b7a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000e9c0 tx=0x7fb1e00e2f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582da663d958a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.442+0000 7fb1dbfff640 1 -- 172.21.15.112:0/2626097616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00df040 con 0x557e91f674d0 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1dbfff640 1 -- 172.21.15.112:0/2626097616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00a6f10 con 0x557e91f674d0 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1dbfff640 1 -- 172.21.15.112:0/2626097616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00a6d50 con 0x557e91f674d0 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2626097616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 msgr2=0x557e9232b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e9232b7a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000e9c0 tx=0x7fb1e00e2f30 comp rx=0 tx=0).stop 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2626097616 shutdown_connections 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f881b0 0x557e9232bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e9232b7a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2626097616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f5f2f0 0x557e92333e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2626097616 >> 172.21.15.112:0/2626097616 conn(0x557e91f7c300 msgr2=0x557e92333710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2626097616 shutdown_connections 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.443+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2626097616 wait complete. 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 -- start start 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 Processor -- start 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e923a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e923a5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92337470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b0410 con 0x557e91f674d0 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b1880 con 0x557e91f881b0 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f01b0ae0 con 0x557e91f5f2f0 2024-04-25T16:49:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1ff61e740 10 client.0 _ll_register_callbacks cb 0x7fb1ec004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e923a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e923a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56706/0 (socket says 172.21.15.112:56706) 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2022135249 learned_addr learned my addr 172.21.15.112:0/2022135249 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f6d0f640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92337470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f650e640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e923a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2022135249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 msgr2=0x557e923a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e923a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 msgr2=0x557e92337470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92337470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.444+0000 7fb1f5d0d640 1 -- 172.21.15.112:0/2022135249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e92337a80 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1f5d0d640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e923a5480 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00a3480 tx=0x7fb1e00e2b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250f90f0adaba055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/2022135249 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e00a6d50 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/2022135249 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fb1e00a2d70 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e00a6f10 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/2022135249 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb1e00a7b00 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 10 client.16281 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557e923ac5f0 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/2022135249 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1e00a6f10 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/2022135249 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fb1e00a7660 con 0x557e91f674d0 2024-04-25T16:49:01.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 10 client.16281 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 10 client.16281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 20 client.16281 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f1c1c0 0x557e923ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 4 client.16281 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e92295ee0 con 0x557e91f1c1c0 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.445+0000 7fb1f650e640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f1c1c0 0x557e923ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.446+0000 7fb1f650e640 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f1c1c0 0x557e923ae880 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7056523625072115 server_cookie=141123dd124ecb1e in_seq=0 out_seq=0 2024-04-25T16:49:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.446+0000 7fb1d9ffb640 10 client.16281 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:01.451 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:01 smithi133 ceph-mon[32691]: pgmap v334: 97 pgs: 97 active+clean; 98 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-04-25T16:49:01.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:01 smithi116 ceph-mon[32712]: pgmap v334: 97 pgs: 97 active+clean; 98 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-04-25T16:49:01.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:01 smithi112 ceph-mon[29122]: pgmap v334: 97 pgs: 97 active+clean; 98 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-04-25T16:49:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.719+0000 7fb1d9ffb640 1 -- 172.21.15.112:0/2022135249 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e92295ee0 con 0x557e91f1c1c0 2024-04-25T16:49:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.719+0000 7fb1d9ffb640 10 client.16281 handle_command_reply: tid=0 2024-04-25T16:49:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.719+0000 7fb1ff61e740 1 client.16281 shutdown 2024-04-25T16:49:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 msgr2=0x557e923a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e923a5480 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb1e00a3480 tx=0x7fb1e00e2b00 comp rx=0 tx=0).stop 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 shutdown_connections 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557e91f881b0 0x557e92337470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557e91f674d0 0x557e923a5480 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x557e91f1c1c0 0x557e923ae880 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 --2- 172.21.15.112:0/2022135249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e91f5f2f0 0x557e923a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 >> 172.21.15.112:0/2022135249 conn(0x557e91f7c300 msgr2=0x557e92329d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1d9ffb640 0 client.16281 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 shutdown_connections 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 1 -- 172.21.15.112:0/2022135249 wait complete. 2024-04-25T16:49:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.720+0000 7fb1ff61e740 20 client.16281 trim_cache size 0 max 16384 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb1dc0afcd0 msgr2=0x7fb1dc0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb1dc0afcd0 0x7fb1dc0b21e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557e92324c60 tx=0x7fb1e80095c0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 msgr2=0x7fb1f00fec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 0x7fb1f00fec80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000e9f0 tx=0x7fb1e000eeb0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 shutdown_connections 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fb1dc0afcd0 0x7fb1dc0b21e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1f0108f30 0x7fb1f00ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1f0108a90 0x7fb1f00fec80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 --2- 172.21.15.112:0/301930849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1f01031e0 0x7fb1f00fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 >> 172.21.15.112:0/301930849 conn(0x7fb1f00fe260 msgr2=0x7fb1f0075170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 shutdown_connections 2024-04-25T16:49:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:01.721+0000 7fb1f8772640 1 -- 172.21.15.112:0/301930849 wait complete. 2024-04-25T16:49:01.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1119966 10.44 MB/sec execute 427 sec latency 107.725 ms 2024-04-25T16:49:02.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1122402 10.46 MB/sec execute 428 sec latency 24.466 ms 2024-04-25T16:49:03.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:03 smithi133 ceph-mon[32691]: pgmap v335: 97 pgs: 97 active+clean; 98 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2024-04-25T16:49:03.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:03 smithi116 ceph-mon[32712]: pgmap v335: 97 pgs: 97 active+clean; 98 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2024-04-25T16:49:03.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:03 smithi112 ceph-mon[29122]: pgmap v335: 97 pgs: 97 active+clean; 98 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 168 op/s 2024-04-25T16:49:03.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1123218 10.44 MB/sec execute 429 sec latency 90.060 ms 2024-04-25T16:49:04.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1125182 10.44 MB/sec execute 430 sec latency 19.431 ms 2024-04-25T16:49:05.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:05 smithi133 ceph-mon[32691]: pgmap v336: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-04-25T16:49:05.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:05 smithi116 ceph-mon[32712]: pgmap v336: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-04-25T16:49:05.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:05 smithi112 ceph-mon[29122]: pgmap v336: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.9 MiB/s wr, 346 op/s 2024-04-25T16:49:05.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1126501 10.43 MB/sec execute 431 sec latency 16.999 ms 2024-04-25T16:49:06.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1127391 10.42 MB/sec execute 432 sec latency 38.684 ms 2024-04-25T16:49:07.397 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:07 smithi112 ceph-mon[29122]: pgmap v337: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-04-25T16:49:07.397 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:07 smithi112 ceph-mon[29122]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T16:49:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:07 smithi133 ceph-mon[32691]: pgmap v337: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-04-25T16:49:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:07 smithi133 ceph-mon[32691]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T16:49:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:07 smithi116 ceph-mon[32712]: pgmap v337: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-04-25T16:49:07.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:07 smithi116 ceph-mon[32712]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T16:49:07.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1127968 10.40 MB/sec execute 433 sec latency 49.151 ms 2024-04-25T16:49:08.891 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1129514 10.40 MB/sec execute 434 sec latency 120.339 ms 2024-04-25T16:49:09.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:09 smithi133 ceph-mon[32691]: pgmap v339: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 310 op/s 2024-04-25T16:49:09.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:09 smithi116 ceph-mon[32712]: pgmap v339: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 310 op/s 2024-04-25T16:49:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:09 smithi112 ceph-mon[29122]: pgmap v339: 97 pgs: 97 active+clean; 95 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.2 MiB/s wr, 310 op/s 2024-04-25T16:49:09.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1130538 10.38 MB/sec execute 435 sec latency 25.407 ms 2024-04-25T16:49:10.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1133072 10.40 MB/sec execute 436 sec latency 28.187 ms 2024-04-25T16:49:11.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:11 smithi133 ceph-mon[32691]: pgmap v340: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 81 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-04-25T16:49:11.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:11 smithi116 ceph-mon[32712]: pgmap v340: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 81 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-04-25T16:49:11.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:11 smithi112 ceph-mon[29122]: pgmap v340: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 81 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-04-25T16:49:11.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1134023 10.38 MB/sec execute 437 sec latency 21.017 ms 2024-04-25T16:49:12.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1136479 10.39 MB/sec execute 438 sec latency 26.656 ms 2024-04-25T16:49:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:13 smithi133 ceph-mon[32691]: pgmap v341: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 81 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-04-25T16:49:13.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:13 smithi116 ceph-mon[32712]: pgmap v341: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 81 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-04-25T16:49:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:13 smithi112 ceph-mon[29122]: pgmap v341: 97 pgs: 3 active+clean+snaptrim, 94 active+clean; 81 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 290 op/s 2024-04-25T16:49:13.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1137265 10.37 MB/sec execute 439 sec latency 139.578 ms 2024-04-25T16:49:14.825 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/1174524129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e410e0d0 msgr2=0x7fc0e41105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/1174524129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e410e0d0 0x7fc0e41105c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009ca0 tx=0x7fc0dc02a950 comp rx=0 tx=0).stop 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/1174524129 shutdown_connections 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/1174524129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e410b770 0x7fc0e410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/1174524129 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e410e0d0 0x7fc0e41105c0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/1174524129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0e4104d10 0x7fc0e4105120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/1174524129 >> 172.21.15.112:0/1174524129 conn(0x7fc0e40781b0 msgr2=0x7fc0e40785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/1174524129 shutdown_connections 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.825+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/1174524129 wait complete. 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 Processor -- start 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 -- start start 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 0x7fc0e41025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.826 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e410b770 0x7fc0e4102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0e410e0d0 0x7fc0e4103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4113d30 con 0x7fc0e4104d10 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4113bb0 con 0x7fc0e410e0d0 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e8b3f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e4113eb0 con 0x7fc0e410b770 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 0x7fc0e41025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 0x7fc0e41025d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58806/0 (socket says 172.21.15.112:58806) 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 -- 172.21.15.112:0/3451494187 learned_addr learned my addr 172.21.15.112:0/3451494187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0e410e0d0 0x7fc0e4103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e410b770 0x7fc0e4102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 -- 172.21.15.112:0/3451494187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e410b770 msgr2=0x7fc0e4102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e410b770 0x7fc0e4102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.827 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 -- 172.21.15.112:0/3451494187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0e410e0d0 msgr2=0x7fc0e4103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0e410e0d0 0x7fc0e4103050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.826+0000 7fc0e259c640 1 -- 172.21.15.112:0/3451494187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e41a8a70 con 0x7fc0e4104d10 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.827+0000 7fc0e259c640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 0x7fc0e41025d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9f0 tx=0x7fc0d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.827+0000 7fc0db7fe640 1 -- 172.21.15.112:0/3451494187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d400cd80 con 0x7fc0e4104d10 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.827+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e41a8d60 con 0x7fc0e4104d10 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.827+0000 7fc0db7fe640 1 -- 172.21.15.112:0/3451494187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0d40108e0 con 0x7fc0e4104d10 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.827+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e41a9210 con 0x7fc0e4104d10 2024-04-25T16:49:14.828 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.827+0000 7fc0db7fe640 1 -- 172.21.15.112:0/3451494187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d4018920 con 0x7fc0e4104d10 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.829+0000 7fc0ef9f3740 1 Processor -- start 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.829+0000 7fc0db7fe640 1 -- 172.21.15.112:0/3451494187 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fc0d4010430 con 0x7fc0e4104d10 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 -- start start 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f3c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f3c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55866f32d460 con 0x55866effe2f0 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55866f2e7590 con 0x55866f0271b0 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55866f329420 con 0x55866f0064d0 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f3c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.836 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:51466/0 (socket says 172.21.15.112:51466) 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f3c7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58814/0 (socket says 172.21.15.112:58814) 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/3660503694 learned_addr learned my addr 172.21.15.112:0/3660503694 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f3c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0db7fe640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc0d00afc50 0x7fc0d00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/3660503694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 msgr2=0x55866f3c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f3c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/3660503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 msgr2=0x55866f3c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f3c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/3660503694 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55866f2f40d0 con 0x55866f0271b0 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0e259c640 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f3c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.830+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f3c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:14.837 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.831+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc0d00afc50 0x7fc0d00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.831+0000 7fc0db7fe640 1 -- 172.21.15.112:0/3451494187 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7fc0d40df3a0 con 0x7fc0e4104d10 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.831+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3c6af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc035000 tx=0x7fc0dc009470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4c3a4155d883beb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.831+0000 7fc0d8ff9640 1 -- 172.21.15.112:0/3660503694 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc034040 con 0x55866f0271b0 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.831+0000 7fc0d8ff9640 1 -- 172.21.15.112:0/3660503694 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc003b70 con 0x55866f0271b0 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.831+0000 7fc0d8ff9640 1 -- 172.21.15.112:0/3660503694 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc030640 con 0x55866f0271b0 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/3660503694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 msgr2=0x55866f3c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:14.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3c6af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc035000 tx=0x7fc0dc009470 comp rx=0 tx=0).stop 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc0d00afc50 0x7fc0d00b2160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc04d000 tx=0x7fc0dc04a000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/3660503694 shutdown_connections 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f3c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f3c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/3660503694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3c6af0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/3660503694 >> 172.21.15.112:0/3660503694 conn(0x55866f01b300 msgr2=0x55866f3c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/3660503694 shutdown_connections 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/3660503694 wait complete. 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.832+0000 7fc0ef9f3740 1 -- start start 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 Processor -- start 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3d7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55866f3d3c20 con 0x55866f0271b0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55866f2b4b60 con 0x55866f0064d0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55866f3d3da0 con 0x55866effe2f0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0ef9f3740 10 client.0 _ll_register_callbacks cb 0x55866f2d92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3d7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58818/0 (socket says 172.21.15.112:58818) 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/2171737681 learned_addr learned my addr 172.21.15.112:0/2171737681 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/2171737681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 msgr2=0x55866f3d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 msgr2=0x55866f3d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/2171737681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55866f03f210 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.833+0000 7fc0e259c640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3d7180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0094e0 tx=0x7fc0dc004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff323124f3abfb99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2171737681 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc009800 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2171737681 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7fc0dc003970 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0dc003b70 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2171737681 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc0dc030ab0 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 10 client.16293 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55866f10b620 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2171737681 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc003b70 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2171737681 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fc0dc046470 con 0x55866f0271b0 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 10 client.16293 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:14.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 20 client.16293 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efe5d80 0x55866f3e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 4 client.16293 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55866f33b070 con 0x55866efe5d80 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.834+0000 7fc0e259c640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efe5d80 0x55866f3e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.835+0000 7fc0c9ffb640 10 client.16293 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.835+0000 7fc0e259c640 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efe5d80 0x55866f3e3600 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded256f60b50bbce server_cookie=89205f3db41a587a in_seq=0 out_seq=0 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.835+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2171737681 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55866f33b070 con 0x55866efe5d80 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.835+0000 7fc0c9ffb640 10 client.16293 handle_command_reply: tid=0 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.835+0000 7fc0ef9f3740 1 client.16293 shutdown 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 msgr2=0x55866f3d7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3d7180 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc0094e0 tx=0x7fc0dc004900 comp rx=0 tx=0).stop 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 shutdown_connections 2024-04-25T16:49:14.842 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3d7180 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efe5d80 0x55866f3e3600 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2171737681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0c9ffb640 0 client.16293 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 >> 172.21.15.112:0/2171737681 conn(0x55866f01b300 msgr2=0x55866f3c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 shutdown_connections 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2171737681 wait complete. 2024-04-25T16:49:14.843 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.836+0000 7fc0ef9f3740 20 client.16293 trim_cache size 0 max 16384 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.851+0000 7fc0ef9f3740 1 Processor -- start 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.851+0000 7fc0ef9f3740 1 -- start start 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41b46e0 con 0x55866f0271b0 2024-04-25T16:49:14.855 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41b1fb0 con 0x55866f0064d0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41afe20 con 0x55866effe2f0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58828/0 (socket says 172.21.15.112:58828) 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/4049857896 learned_addr learned my addr 172.21.15.112:0/4049857896 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e259c640 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/4049857896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 msgr2=0x55866f3caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/4049857896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 msgr2=0x55866f3c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 -- 172.21.15.112:0/4049857896 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55866f3c3e90 con 0x55866f0271b0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3c6dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009a60 tx=0x7fc0dc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ca2365a68eeab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.852+0000 7fc0cbfff640 1 -- 172.21.15.112:0/4049857896 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc009800 con 0x55866f0271b0 2024-04-25T16:49:14.856 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0cbfff640 1 -- 172.21.15.112:0/4049857896 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc046e50 con 0x55866f0271b0 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0cbfff640 1 -- 172.21.15.112:0/4049857896 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0dc046a80 con 0x55866f0271b0 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/4049857896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 msgr2=0x55866f3c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3c6dc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009a60 tx=0x7fc0dc02a950 comp rx=0 tx=0).stop 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/4049857896 shutdown_connections 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866f0271b0 0x55866f3c6dc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0064d0 0x55866f3c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/4049857896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866effe2f0 0x55866f3caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/4049857896 >> 172.21.15.112:0/4049857896 conn(0x55866f01b300 msgr2=0x55866f3ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/4049857896 shutdown_connections 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.853+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/4049857896 wait complete. 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 -- start start 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 Processor -- start 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f443e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f4443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3dcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41b0470 con 0x55866effe2f0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41b18e0 con 0x55866f0271b0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e41b0b40 con 0x55866f0064d0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0ef9f3740 10 client.0 _ll_register_callbacks cb 0x7fc0d40e2dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f443e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f443e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58832/0 (socket says 172.21.15.112:58832) 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 -- 172.21.15.112:0/2184204631 learned_addr learned my addr 172.21.15.112:0/2184204631 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f4443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e2d9d640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3dcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 -- 172.21.15.112:0/2184204631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 msgr2=0x55866f4443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f4443d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 -- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 msgr2=0x55866f3dcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3dcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.854+0000 7fc0e259c640 1 -- 172.21.15.112:0/2184204631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55866f3dd3e0 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0e259c640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f443e90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40a35f0 tx=0x7fc0d4018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2319d4695d29c8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2184204631 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d40a69d0 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2184204631 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8644+0+0 (secure 0 0 0) 0x7fc0d40a2ee0 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0dc046e50 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2184204631 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc0d4010720 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 10 client.16299 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55866f44b5d0 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2184204631 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0dc046e50 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2184204631 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fc0d40a30e0 con 0x55866effe2f0 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 10 client.16299 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 20 client.16299 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efbb1c0 0x55866f44dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 4 client.16299 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:49:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55866f334ee0 con 0x55866efbb1c0 2024-04-25T16:49:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.855+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efbb1c0 0x55866f44dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.856+0000 7fc0dbfff640 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efbb1c0 0x55866f44dd90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=415071cc0b7d9290 server_cookie=db8476f7bd2ebd59 in_seq=0 out_seq=0 2024-04-25T16:49:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:14.856+0000 7fc0c9ffb640 10 client.16299 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:14.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1139219 10.37 MB/sec execute 440 sec latency 29.876 ms 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.034+0000 7fc0c9ffb640 1 -- 172.21.15.112:0/2184204631 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55866f334ee0 con 0x55866efbb1c0 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.034+0000 7fc0c9ffb640 10 client.16299 handle_command_reply: tid=0 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.034+0000 7fc0ef9f3740 1 client.16299 shutdown 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 msgr2=0x55866f443e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f443e90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40a35f0 tx=0x7fc0d4018420 comp rx=0 tx=0).stop 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 shutdown_connections 2024-04-25T16:49:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55866efbb1c0 0x55866f44dd90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0c9ffb640 0 client.16299 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55866f0064d0 0x55866f4443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55866f0271b0 0x55866f3dcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 --2- 172.21.15.112:0/2184204631 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55866effe2f0 0x55866f443e90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 >> 172.21.15.112:0/2184204631 conn(0x55866f01b300 msgr2=0x55866f3c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 shutdown_connections 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 1 -- 172.21.15.112:0/2184204631 wait complete. 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.035+0000 7fc0ef9f3740 20 client.16299 trim_cache size 0 max 16384 2024-04-25T16:49:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:49:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:49:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:49:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:49:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc0d00afc50 msgr2=0x7fc0d00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc0d00afc50 0x7fc0d00b2160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc04d000 tx=0x7fc0dc04a000 comp rx=0 tx=0).stop 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 msgr2=0x7fc0e41025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 0x7fc0e41025d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc0d400e9f0 tx=0x7fc0d400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 shutdown_connections 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc0d00afc50 0x7fc0d00b2160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc0e410b770 0x7fc0e4102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0e410e0d0 0x7fc0e4103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 --2- 172.21.15.112:0/3451494187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc0e4104d10 0x7fc0e41025d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 >> 172.21.15.112:0/3451494187 conn(0x7fc0e40781b0 msgr2=0x7fc0e4101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 shutdown_connections 2024-04-25T16:49:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:15.036+0000 7fc0e8b3f640 1 -- 172.21.15.112:0/3451494187 wait complete. 2024-04-25T16:49:15.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:15 smithi133 ceph-mon[32691]: pgmap v342: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:15.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:15 smithi116 ceph-mon[32712]: pgmap v342: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:15 smithi112 ceph-mon[29122]: pgmap v342: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:15.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1140562 10.37 MB/sec execute 441 sec latency 99.809 ms 2024-04-25T16:49:16.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1142498 10.36 MB/sec execute 442 sec latency 27.602 ms 2024-04-25T16:49:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:17 smithi133 ceph-mon[32691]: pgmap v343: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:17 smithi133 ceph-mon[32691]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T16:49:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:17 smithi116 ceph-mon[32712]: pgmap v343: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:17 smithi116 ceph-mon[32712]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T16:49:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:17 smithi112 ceph-mon[29122]: pgmap v343: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:17 smithi112 ceph-mon[29122]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T16:49:17.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1144131 10.36 MB/sec execute 443 sec latency 23.145 ms 2024-04-25T16:49:18.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1145523 10.35 MB/sec execute 444 sec latency 51.991 ms 2024-04-25T16:49:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:19 smithi133 ceph-mon[32691]: pgmap v345: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:19 smithi116 ceph-mon[32712]: pgmap v345: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:19 smithi112 ceph-mon[29122]: pgmap v345: 97 pgs: 97 active+clean; 86 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-04-25T16:49:19.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1147421 10.36 MB/sec execute 445 sec latency 18.015 ms 2024-04-25T16:49:20.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1148851 10.35 MB/sec execute 446 sec latency 17.644 ms 2024-04-25T16:49:21.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:21 smithi133 ceph-mon[32691]: pgmap v346: 97 pgs: 97 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2024-04-25T16:49:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:21 smithi116 ceph-mon[32712]: pgmap v346: 97 pgs: 97 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2024-04-25T16:49:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:21 smithi112 ceph-mon[29122]: pgmap v346: 97 pgs: 97 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2024-04-25T16:49:21.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1150926 10.35 MB/sec execute 447 sec latency 24.170 ms 2024-04-25T16:49:22.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1152237 10.35 MB/sec execute 448 sec latency 16.472 ms 2024-04-25T16:49:23.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:23 smithi133 ceph-mon[32691]: pgmap v347: 97 pgs: 97 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2024-04-25T16:49:23.510 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:23 smithi116 ceph-mon[32712]: pgmap v347: 97 pgs: 97 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2024-04-25T16:49:23.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:23 smithi112 ceph-mon[29122]: pgmap v347: 97 pgs: 97 active+clean; 84 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 294 op/s 2024-04-25T16:49:23.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1154198 10.34 MB/sec execute 449 sec latency 43.210 ms 2024-04-25T16:49:24.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1155175 10.33 MB/sec execute 450 sec latency 35.064 ms 2024-04-25T16:49:25.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:25 smithi133 ceph-mon[32691]: pgmap v348: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-25T16:49:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:25 smithi116 ceph-mon[32712]: pgmap v348: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-25T16:49:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:25 smithi112 ceph-mon[29122]: pgmap v348: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-25T16:49:25.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1157365 10.34 MB/sec execute 451 sec latency 21.888 ms 2024-04-25T16:49:26.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1158637 10.33 MB/sec execute 452 sec latency 20.151 ms 2024-04-25T16:49:27.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:27 smithi133 ceph-mon[32691]: pgmap v349: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-25T16:49:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:27 smithi116 ceph-mon[32712]: pgmap v349: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-25T16:49:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:27 smithi112 ceph-mon[29122]: pgmap v349: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-04-25T16:49:27.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1160846 10.33 MB/sec execute 453 sec latency 25.546 ms 2024-04-25T16:49:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 -- 172.21.15.112:0/486850134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81031e0 msgr2=0x7f00e810f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 --2- 172.21.15.112:0/486850134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81031e0 0x7f00e810f5b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009ca0 tx=0x7f00e002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 -- 172.21.15.112:0/486850134 shutdown_connections 2024-04-25T16:49:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 --2- 172.21.15.112:0/486850134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81031e0 0x7f00e810f5b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 --2- 172.21.15.112:0/486850134 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 0x7f00e8102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 --2- 172.21.15.112:0/486850134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00e8108a90 0x7f00e8102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 -- 172.21.15.112:0/486850134 >> 172.21.15.112:0/486850134 conn(0x7f00e8078080 msgr2=0x7f00e80784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 -- 172.21.15.112:0/486850134 shutdown_connections 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.141+0000 7f00ef172640 1 -- 172.21.15.112:0/486850134 wait complete. 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 Processor -- start 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 -- start start 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00e81031e0 0x7f00e819b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 0x7f00e819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 0x7f00e81a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112ca0 con 0x7f00e8108a90 2024-04-25T16:49:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112b20 con 0x7f00e8108f30 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ef172640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8112e20 con 0x7f00e81031e0 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 0x7f00e819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 0x7f00e819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50028/0 (socket says 172.21.15.112:50028) 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ed70f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 0x7f00e81a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 -- 172.21.15.112:0/3925886089 learned_addr learned my addr 172.21.15.112:0/3925886089 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00e81031e0 0x7f00e819b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 -- 172.21.15.112:0/3925886089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00e81031e0 msgr2=0x7f00e819b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00e81031e0 0x7f00e819b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 -- 172.21.15.112:0/3925886089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 msgr2=0x7f00e81a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 0x7f00e81a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00e7fff640 1 -- 172.21.15.112:0/3925886089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e819c340 con 0x7f00e8108a90 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.142+0000 7f00ed70f640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 0x7f00e81a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.143+0000 7f00e7fff640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 0x7f00e819bac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e9f0 tx=0x7f00d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.143+0000 7f00e5ffb640 1 -- 172.21.15.112:0/3925886089 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400ccc0 con 0x7f00e8108a90 2024-04-25T16:49:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.143+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81a6c00 con 0x7f00e8108a90 2024-04-25T16:49:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.143+0000 7f00e5ffb640 1 -- 172.21.15.112:0/3925886089 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00d40035f0 con 0x7f00e8108a90 2024-04-25T16:49:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.143+0000 7f00e5ffb640 1 -- 172.21.15.112:0/3925886089 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d400ce60 con 0x7f00e8108a90 2024-04-25T16:49:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.143+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a7150 con 0x7f00e8108a90 2024-04-25T16:49:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.145+0000 7f00e5ffb640 1 -- 172.21.15.112:0/3925886089 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f00d4003aa0 con 0x7f00e8108a90 2024-04-25T16:49:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.145+0000 7f00f6029740 1 Processor -- start 2024-04-25T16:49:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.145+0000 7f00f6029740 1 -- start start 2024-04-25T16:49:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00f6029740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22aa3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00f6029740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aa4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00f6029740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aa4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00f6029740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f22a0a460 con 0x559f226e34d0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00f6029740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559f229c4590 con 0x559f226db2f0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00f6029740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f22a06420 con 0x559f227041b0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00e5ffb640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f00d80afc50 0x7f00d80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aa4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aa4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ed70f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22aa3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aa4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:53840/0 (socket says 172.21.15.112:53840) 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 -- 172.21.15.112:0/2632592995 learned_addr learned my addr 172.21.15.112:0/2632592995 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ed70f640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f00d80afc50 0x7f00d80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 -- 172.21.15.112:0/2632592995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 msgr2=0x559f22aa3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22aa3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 -- 172.21.15.112:0/2632592995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 msgr2=0x559f22aa4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00e5ffb640 1 -- 172.21.15.112:0/3925886089 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f00d40df3b0 con 0x7f00e8108a90 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aa4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ecf0e640 1 -- 172.21.15.112:0/2632592995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f229d10d0 con 0x559f226db2f0 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.146+0000 7f00ed70f640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f00d80afc50 0x7f00d80b2160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00e002f500 tx=0x7f00e00095c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.147+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aa4030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009c30 tx=0x7f00dc0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36e4b69ae78ed569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.147+0000 7f00d2ffd640 1 -- 172.21.15.112:0/2632592995 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc012450 con 0x559f226db2f0 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.147+0000 7f00d2ffd640 1 -- 172.21.15.112:0/2632592995 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc012aa0 con 0x559f226db2f0 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.147+0000 7f00d2ffd640 1 -- 172.21.15.112:0/2632592995 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00fa10 con 0x559f226db2f0 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 -- 172.21.15.112:0/2632592995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 msgr2=0x559f22aa4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aa4030 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009c30 tx=0x7f00dc0117b0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 -- 172.21.15.112:0/2632592995 shutdown_connections 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aa4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aa4030 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 --2- 172.21.15.112:0/2632592995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22aa3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 -- 172.21.15.112:0/2632592995 >> 172.21.15.112:0/2632592995 conn(0x559f226f8300 msgr2=0x559f22aa02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 -- 172.21.15.112:0/2632592995 shutdown_connections 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 -- 172.21.15.112:0/2632592995 wait complete. 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 -- start start 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.148+0000 7f00f6029740 1 Processor -- start 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226db2f0 0x559f22aae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ab40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559f22ab0c20 con 0x559f226e34d0 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559f22991b60 con 0x559f227041b0 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f22ab0da0 con 0x559f226db2f0 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00f6029740 10 client.0 _ll_register_callbacks cb 0x559f229b62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50040/0 (socket says 172.21.15.112:50040) 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 -- 172.21.15.112:0/1372906225 learned_addr learned my addr 172.21.15.112:0/1372906225 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00ed70f640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ab40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226db2f0 0x559f22aae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 -- 172.21.15.112:0/1372906225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226db2f0 msgr2=0x559f22aae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226db2f0 0x559f22aae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 -- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 msgr2=0x559f22ab40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ab40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 -- 172.21.15.112:0/1372906225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f2271c210 con 0x559f226e34d0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aae6c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f00d40a7200 tx=0x7f00d40a72d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90146e821ab4a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.149+0000 7f00c7fff640 1 -- 172.21.15.112:0/1372906225 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d40a7960 con 0x559f226e34d0 2024-04-25T16:49:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.150+0000 7f00c7fff640 1 -- 172.21.15.112:0/1372906225 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f00d40afed0 con 0x559f226e34d0 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.150+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00dc012aa0 con 0x559f226e34d0 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.150+0000 7f00c7fff640 1 -- 172.21.15.112:0/1372906225 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00d40a7010 con 0x559f226e34d0 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.150+0000 7f00f6029740 10 client.16311 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.150+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559f227e8620 con 0x559f226e34d0 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.150+0000 7f00c7fff640 1 -- 172.21.15.112:0/1372906225 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc012aa0 con 0x559f226e34d0 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00c7fff640 1 -- 172.21.15.112:0/1372906225 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f00d4003fb0 con 0x559f226e34d0 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00f6029740 10 client.16311 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00f6029740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00f6029740 20 client.16311 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00f6029740 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226c2d80 0x559f22ac0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00f6029740 4 client.16311 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f22a18070 con 0x559f226c2d80 2024-04-25T16:49:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.151+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226c2d80 0x559f22ac0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.153+0000 7f00c7fff640 10 client.16311 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.153+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226c2d80 0x559f22ac0740 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89dc7a59ac598353 server_cookie=67f4318101ed052a in_seq=0 out_seq=0 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00c7fff640 1 -- 172.21.15.112:0/1372906225 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559f22a18070 con 0x559f226c2d80 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00c7fff640 10 client.16311 handle_command_reply: tid=0 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 client.16311 shutdown 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 msgr2=0x559f22aae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aae6c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f00d40a7200 tx=0x7f00d40a72d0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 shutdown_connections 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ab40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22aae6c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226c2d80 0x559f22ac0740 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 --2- 172.21.15.112:0/1372906225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226db2f0 0x559f22aae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00c7fff640 0 client.16311 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 >> 172.21.15.112:0/1372906225 conn(0x559f226f8300 msgr2=0x559f22aa3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 shutdown_connections 2024-04-25T16:49:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.154+0000 7f00f6029740 1 -- 172.21.15.112:0/1372906225 wait complete. 2024-04-25T16:49:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.155+0000 7f00f6029740 20 client.16311 trim_cache size 0 max 16384 2024-04-25T16:49:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.177+0000 7f00f6029740 1 Processor -- start 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.177+0000 7f00f6029740 1 -- start start 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00f6029740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 0x559f22aa7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00f6029740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226e34d0 0x559f22ac5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00f6029740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ac6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00f6029740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81b2450 con 0x559f226db2f0 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00f6029740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81afd20 con 0x559f227041b0 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00f6029740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81adb90 con 0x559f226e34d0 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 0x559f22aa7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 0x559f22aa7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50052/0 (socket says 172.21.15.112:50052) 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 -- 172.21.15.112:0/1994236130 learned_addr learned my addr 172.21.15.112:0/1994236130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ed70f640 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ac6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226e34d0 0x559f22ac5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 -- 172.21.15.112:0/1994236130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226e34d0 msgr2=0x559f22ac5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226e34d0 0x559f22ac5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 -- 172.21.15.112:0/1994236130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 msgr2=0x559f22ac6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ac6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 -- 172.21.15.112:0/1994236130 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f22aa9b30 con 0x559f226db2f0 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.178+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 0x559f22aa7e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f00dc011cb0 tx=0x7f00dc00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd067647926afc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.179+0000 7f00d27fc640 1 -- 172.21.15.112:0/1994236130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc033070 con 0x559f226db2f0 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.179+0000 7f00d27fc640 1 -- 172.21.15.112:0/1994236130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc0056f0 con 0x559f226db2f0 2024-04-25T16:49:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.179+0000 7f00d27fc640 1 -- 172.21.15.112:0/1994236130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00fe20 con 0x559f226db2f0 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 -- 172.21.15.112:0/1994236130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 msgr2=0x559f22aa7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 0x559f22aa7e50 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f00dc011cb0 tx=0x7f00dc00f5c0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 -- 172.21.15.112:0/1994236130 shutdown_connections 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f226e34d0 0x559f22ac5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f227041b0 0x559f22ac6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 --2- 172.21.15.112:0/1994236130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226db2f0 0x559f22aa7e50 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 -- 172.21.15.112:0/1994236130 >> 172.21.15.112:0/1994236130 conn(0x559f226f8300 msgr2=0x559f22aa7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 -- 172.21.15.112:0/1994236130 shutdown_connections 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 -- 172.21.15.112:0/1994236130 wait complete. 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 -- start start 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.180+0000 7f00f6029740 1 Processor -- start 2024-04-25T16:49:28.184 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aba8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22ab7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22ab8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81ae1e0 con 0x559f226e34d0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81af650 con 0x559f226db2f0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00e81ae8b0 con 0x559f227041b0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00f6029740 10 client.0 _ll_register_callbacks cb 0x7f00d40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22ab7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00ecf0e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aba8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00ed70f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22ab8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22ab7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50066/0 (socket says 172.21.15.112:50066) 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00ed70f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22ab8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:40658/0 (socket says 172.21.15.112:40658) 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00ecf0e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aba8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:53870/0 (socket says 172.21.15.112:53870) 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 -- 172.21.15.112:0/1591668221 learned_addr learned my addr 172.21.15.112:0/1591668221 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 -- 172.21.15.112:0/1591668221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 msgr2=0x559f22ab8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.185 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22ab8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 -- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 msgr2=0x559f22aba8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aba8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 -- 172.21.15.112:0/1591668221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f22ab8820 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00e7fff640 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22ab7c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f00d4003eb0 tx=0x7f00d40a7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d1df84620dda4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.181+0000 7f00c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00c7fff640 1 -- 172.21.15.112:0/1591668221 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d40a6e50 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00c7fff640 1 -- 172.21.15.112:0/1591668221 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f00d40a6760 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00dc0056f0 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00c7fff640 1 -- 172.21.15.112:0/1591668221 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f00d40b0390 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 10 client.16317 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559f22b288d0 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00c7fff640 1 -- 172.21.15.112:0/1591668221 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00dc0056f0 con 0x559f226e34d0 2024-04-25T16:49:28.186 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00c7fff640 1 -- 172.21.15.112:0/1591668221 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f00d40c8400 con 0x559f226e34d0 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 10 client.16317 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 20 client.16317 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226981c0 0x559f22b2b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 4 client.16317 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f22a11ee0 con 0x559f226981c0 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.182+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226981c0 0x559f22b2b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.183+0000 7f00c7fff640 10 client.16317 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:28.187 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.183+0000 7f00ecf0e640 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226981c0 0x559f22b2b030 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3989ea8934e2e691 server_cookie=b2cf2314aed66607 in_seq=0 out_seq=0 2024-04-25T16:49:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.354+0000 7f00c7fff640 1 -- 172.21.15.112:0/1591668221 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f22a11ee0 con 0x559f226981c0 2024-04-25T16:49:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.354+0000 7f00c7fff640 10 client.16317 handle_command_reply: tid=0 2024-04-25T16:49:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.354+0000 7f00f6029740 1 client.16317 shutdown 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 msgr2=0x559f22ab7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22ab7c40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f00d4003eb0 tx=0x7f00d40a7970 comp rx=0 tx=0).stop 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 shutdown_connections 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559f226981c0 0x559f22b2b030 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00c7fff640 0 client.16317 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f227041b0 0x559f22ab8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559f226e34d0 0x559f22ab7c40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 --2- 172.21.15.112:0/1591668221 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559f226db2f0 0x559f22aba8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 >> 172.21.15.112:0/1591668221 conn(0x559f226f8300 msgr2=0x559f22aa3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 shutdown_connections 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 1 -- 172.21.15.112:0/1591668221 wait complete. 2024-04-25T16:49:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.355+0000 7f00f6029740 20 client.16317 trim_cache size 0 max 16384 2024-04-25T16:49:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:49:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:49:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:49:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f00d80afc50 msgr2=0x7f00d80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f00d80afc50 0x7f00d80b2160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f00e002f500 tx=0x7f00e00095c0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 msgr2=0x7f00e819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 0x7f00e819bac0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e9f0 tx=0x7f00d400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 shutdown_connections 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f00d80afc50 0x7f00d80b2160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00e8108f30 0x7f00e81a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e8108a90 0x7f00e819bac0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 --2- 172.21.15.112:0/3925886089 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00e81031e0 0x7f00e819b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 >> 172.21.15.112:0/3925886089 conn(0x7f00e8078080 msgr2=0x7f00e80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 shutdown_connections 2024-04-25T16:49:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:28.356+0000 7f00ef172640 1 -- 172.21.15.112:0/3925886089 wait complete. 2024-04-25T16:49:28.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1161767 10.32 MB/sec execute 454 sec latency 111.816 ms 2024-04-25T16:49:29.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:29 smithi133 ceph-mon[32691]: pgmap v350: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2024-04-25T16:49:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:29 smithi116 ceph-mon[32712]: pgmap v350: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2024-04-25T16:49:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:29 smithi112 ceph-mon[29122]: pgmap v350: 97 pgs: 97 active+clean; 88 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 279 op/s 2024-04-25T16:49:29.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1163250 10.31 MB/sec execute 455 sec latency 44.961 ms 2024-04-25T16:49:30.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1165178 10.32 MB/sec execute 456 sec latency 17.877 ms 2024-04-25T16:49:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:31 smithi133 ceph-mon[32691]: pgmap v351: 97 pgs: 97 active+clean; 75 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-04-25T16:49:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:31 smithi116 ceph-mon[32712]: pgmap v351: 97 pgs: 97 active+clean; 75 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-04-25T16:49:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:31 smithi112 ceph-mon[29122]: pgmap v351: 97 pgs: 97 active+clean; 75 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 399 op/s 2024-04-25T16:49:31.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1166676 10.31 MB/sec execute 457 sec latency 15.748 ms 2024-04-25T16:49:32.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1168722 10.31 MB/sec execute 458 sec latency 15.320 ms 2024-04-25T16:49:33.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:33 smithi133 ceph-mon[32691]: pgmap v352: 97 pgs: 97 active+clean; 75 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-04-25T16:49:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:33 smithi116 ceph-mon[32712]: pgmap v352: 97 pgs: 97 active+clean; 75 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-04-25T16:49:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:33 smithi112 ceph-mon[29122]: pgmap v352: 97 pgs: 97 active+clean; 75 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-04-25T16:49:33.892 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1170127 10.31 MB/sec execute 459 sec latency 78.049 ms 2024-04-25T16:49:34.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1172228 10.31 MB/sec execute 460 sec latency 25.293 ms 2024-04-25T16:49:35.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:35 smithi133 ceph-mon[32691]: pgmap v353: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 312 op/s 2024-04-25T16:49:35.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:35 smithi116 ceph-mon[32712]: pgmap v353: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 312 op/s 2024-04-25T16:49:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:35 smithi112 ceph-mon[29122]: pgmap v353: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.3 MiB/s wr, 312 op/s 2024-04-25T16:49:35.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1173718 10.30 MB/sec execute 461 sec latency 15.664 ms 2024-04-25T16:49:36.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1175787 10.31 MB/sec execute 462 sec latency 18.630 ms 2024-04-25T16:49:37.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:37 smithi133 ceph-mon[32691]: pgmap v354: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-04-25T16:49:37.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:37 smithi116 ceph-mon[32712]: pgmap v354: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-04-25T16:49:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:37 smithi112 ceph-mon[29122]: pgmap v354: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-04-25T16:49:37.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1177349 10.30 MB/sec execute 463 sec latency 16.576 ms 2024-04-25T16:49:38.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1179320 10.30 MB/sec execute 464 sec latency 63.315 ms 2024-04-25T16:49:39.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:39 smithi133 ceph-mon[32691]: pgmap v355: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-04-25T16:49:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:39 smithi116 ceph-mon[32712]: pgmap v355: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-04-25T16:49:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:39 smithi112 ceph-mon[29122]: pgmap v355: 97 pgs: 97 active+clean; 77 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-04-25T16:49:39.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1180730 10.30 MB/sec execute 465 sec latency 84.220 ms 2024-04-25T16:49:40.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1182865 10.30 MB/sec execute 466 sec latency 15.450 ms 2024-04-25T16:49:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:41 smithi133 ceph-mon[32691]: pgmap v356: 97 pgs: 97 active+clean; 83 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 -- 172.21.15.112:0/4005191806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 msgr2=0x7fae74103740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 --2- 172.21.15.112:0/4005191806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae74103740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009ca0 tx=0x7fae6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 -- 172.21.15.112:0/4005191806 shutdown_connections 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 --2- 172.21.15.112:0/4005191806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae741042f0 0x7fae741106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 --2- 172.21.15.112:0/4005191806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae7410a040 0x7fae74103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 --2- 172.21.15.112:0/4005191806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae74103740 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 -- 172.21.15.112:0/4005191806 >> 172.21.15.112:0/4005191806 conn(0x7fae740792a0 msgr2=0x7fae740796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 -- 172.21.15.112:0/4005191806 shutdown_connections 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 -- 172.21.15.112:0/4005191806 wait complete. 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.463+0000 7fae7bce7640 1 Processor -- start 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 -- start start 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae741042f0 0x7fae7419a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae7419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7410a040 0x7fae7419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae74113d90 con 0x7fae74109ba0 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae74113c10 con 0x7fae741042f0 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7bce7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae74113f10 con 0x7fae7410a040 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae7419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae7419ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40950/0 (socket says 172.21.15.112:40950) 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 -- 172.21.15.112:0/1484892994 learned_addr learned my addr 172.21.15.112:0/1484892994 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:41.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79a83640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae741042f0 0x7fae7419a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae7a284640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7410a040 0x7fae7419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 -- 172.21.15.112:0/1484892994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7410a040 msgr2=0x7fae7419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7410a040 0x7fae7419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 -- 172.21.15.112:0/1484892994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae741042f0 msgr2=0x7fae7419a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae741042f0 0x7fae7419a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 -- 172.21.15.112:0/1484892994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7419f610 con 0x7fae74109ba0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.464+0000 7fae79282640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae7419ad10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e9f0 tx=0x7fae6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.465+0000 7fae6affd640 1 -- 172.21.15.112:0/1484892994 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400ccc0 con 0x7fae74109ba0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.465+0000 7fae6affd640 1 -- 172.21.15.112:0/1484892994 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae640035f0 con 0x7fae74109ba0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.465+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7419f900 con 0x7fae74109ba0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.465+0000 7fae6affd640 1 -- 172.21.15.112:0/1484892994 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400ce60 con 0x7fae74109ba0 2024-04-25T16:49:41.465 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.465+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae741a7280 con 0x7fae74109ba0 2024-04-25T16:49:41.471 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.467+0000 7fae6affd640 1 -- 172.21.15.112:0/1484892994 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fae64003aa0 con 0x7fae74109ba0 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.467+0000 7fae82b98740 1 Processor -- start 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 -- start start 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae6affd640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fae600afc50 0x7fae600b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fe51b0 0x55e0e9384980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9384ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e9385400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.472 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0e92eb460 con 0x55e0e8fc44d0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e0e92a5590 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae82b98740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e0e92e7420 con 0x55e0e8fe51b0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae7a284640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fae600afc50 0x7fae600b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9384ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9384ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:44840/0 (socket says 172.21.15.112:44840) 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 -- 172.21.15.112:0/826541313 learned_addr learned my addr 172.21.15.112:0/826541313 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae7a284640 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e9385400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae6affd640 1 -- 172.21.15.112:0/1484892994 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fae640df300 con 0x7fae74109ba0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79a83640 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fe51b0 0x55e0e9384980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 -- 172.21.15.112:0/826541313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fe51b0 msgr2=0x55e0e9384980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fe51b0 0x55e0e9384980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.473 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 -- 172.21.15.112:0/826541313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 msgr2=0x55e0e9385400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e9385400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.468+0000 7fae79282640 1 -- 172.21.15.112:0/826541313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e92b20d0 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.469+0000 7fae79282640 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9384ec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae64004770 tx=0x7fae640a6da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d01c5feab22bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.469+0000 7fae7a284640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fae600afc50 0x7fae600b2160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b590 tx=0x7fae7000baa0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.469+0000 7fae5ffff640 1 -- 172.21.15.112:0/826541313 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640ca070 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.469+0000 7fae5ffff640 1 -- 172.21.15.112:0/826541313 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae640a6f60 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.469+0000 7fae5ffff640 1 -- 172.21.15.112:0/826541313 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640dcc80 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 -- 172.21.15.112:0/826541313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 msgr2=0x55e0e9384ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9384ec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fae64004770 tx=0x7fae640a6da0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 -- 172.21.15.112:0/826541313 shutdown_connections 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e9385400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9384ec0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.474 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 --2- 172.21.15.112:0/826541313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fe51b0 0x55e0e9384980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 -- 172.21.15.112:0/826541313 >> 172.21.15.112:0/826541313 conn(0x55e0e8fd9300 msgr2=0x55e0e93812a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 -- 172.21.15.112:0/826541313 shutdown_connections 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 -- 172.21.15.112:0/826541313 wait complete. 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 -- start start 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.470+0000 7fae82b98740 1 Processor -- start 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e93935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 0x55e0e9393b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 0x55e0e9399520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e0e9396030 con 0x55e0e8fe51b0 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e0e9272b60 con 0x55e0e8fc44d0 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e0e93961b0 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 10 client.0 _ll_register_callbacks cb 0x55e0e92972b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae7a284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 0x55e0e9399520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 0x55e0e9393b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 0x55e0e9393b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:44850/0 (socket says 172.21.15.112:44850) 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 -- 172.21.15.112:0/3399979603 learned_addr learned my addr 172.21.15.112:0/3399979603 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae7a284640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 0x55e0e9399520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40980/0 (socket says 172.21.15.112:40980) 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79a83640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e93935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 -- 172.21.15.112:0/3399979603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 msgr2=0x55e0e93935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e93935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 -- 172.21.15.112:0/3399979603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 msgr2=0x55e0e9399520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 0x55e0e9399520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 -- 172.21.15.112:0/3399979603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e8ffd210 con 0x55e0e8fc44d0 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae7a284640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 0x55e0e9399520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae79282640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 0x55e0e9393b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e9c0 tx=0x7fae640dca20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85fa5cabd7f97fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae5d7fa640 1 -- 172.21.15.112:0/3399979603 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640a7450 con 0x55e0e8fc44d0 2024-04-25T16:49:41.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae640a6f60 con 0x55e0e8fc44d0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.471+0000 7fae5d7fa640 1 -- 172.21.15.112:0/3399979603 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fae640a78c0 con 0x55e0e8fc44d0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae5d7fa640 1 -- 172.21.15.112:0/3399979603 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae640ab800 con 0x55e0e8fc44d0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 10 client.26012 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0e90c9620 con 0x55e0e8fc44d0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae5d7fa640 1 -- 172.21.15.112:0/3399979603 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae640a6f60 con 0x55e0e8fc44d0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae5d7fa640 1 -- 172.21.15.112:0/3399979603 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fae640aeb80 con 0x55e0e8fc44d0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 10 client.26012 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 10 client.26012 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 20 client.26012 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8fa3d80 0x55e0e93a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 4 client.26012 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0e92f9070 con 0x55e0e8fa3d80 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae79a83640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8fa3d80 0x55e0e93a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.472+0000 7fae5d7fa640 10 client.26012 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:41.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.473+0000 7fae79a83640 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8fa3d80 0x55e0e93a5980 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3408576552011c server_cookie=186011c034dafdcd in_seq=0 out_seq=0 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.475+0000 7fae5d7fa640 1 -- 172.21.15.112:0/3399979603 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0e92f9070 con 0x55e0e8fa3d80 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.475+0000 7fae5d7fa640 10 client.26012 handle_command_reply: tid=0 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.475+0000 7fae82b98740 1 client.26012 shutdown 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 msgr2=0x55e0e9393b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 0x55e0e9393b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e9c0 tx=0x7fae640dca20 comp rx=0 tx=0).stop 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 shutdown_connections 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fe51b0 0x55e0e9399520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fc44d0 0x55e0e9393b00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8fa3d80 0x55e0e93a5980 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 --2- 172.21.15.112:0/3399979603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e93935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae5d7fa640 0 client.26012 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 >> 172.21.15.112:0/3399979603 conn(0x55e0e8fd9300 msgr2=0x55e0e93867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 shutdown_connections 2024-04-25T16:49:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 1 -- 172.21.15.112:0/3399979603 wait complete. 2024-04-25T16:49:41.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.476+0000 7fae82b98740 20 client.26012 trim_cache size 0 max 16384 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 Processor -- start 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 -- start start 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9381430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e93993e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e9392b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae741b03c0 con 0x55e0e8fc44d0 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae741adc90 con 0x55e0e8fe51b0 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae82b98740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae741abb00 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e93993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae7a284640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e9392b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e93993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40984/0 (socket says 172.21.15.112:40984) 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 -- 172.21.15.112:0/765632238 learned_addr learned my addr 172.21.15.112:0/765632238 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79a83640 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9381430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 -- 172.21.15.112:0/765632238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 msgr2=0x55e0e9381430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9381430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 -- 172.21.15.112:0/765632238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 msgr2=0x55e0e9392b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e9392b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.495+0000 7fae79282640 1 -- 172.21.15.112:0/765632238 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0e9381e90 con 0x55e0e8fc44d0 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.496+0000 7fae79282640 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e93993e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fae640a46c0 tx=0x7fae64090050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad249b974835fd70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.496+0000 7fae5f7fe640 1 -- 172.21.15.112:0/765632238 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640a6f60 con 0x55e0e8fc44d0 2024-04-25T16:49:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.496+0000 7fae5f7fe640 1 -- 172.21.15.112:0/765632238 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae640a7450 con 0x55e0e8fc44d0 2024-04-25T16:49:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.496+0000 7fae5f7fe640 1 -- 172.21.15.112:0/765632238 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae640ca390 con 0x55e0e8fc44d0 2024-04-25T16:49:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 -- 172.21.15.112:0/765632238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 msgr2=0x55e0e93993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e93993e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fae640a46c0 tx=0x7fae64090050 comp rx=0 tx=0).stop 2024-04-25T16:49:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 -- 172.21.15.112:0/765632238 shutdown_connections 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e9392b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fc44d0 0x55e0e93993e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 --2- 172.21.15.112:0/765632238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fbc2f0 0x55e0e9381430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 -- 172.21.15.112:0/765632238 >> 172.21.15.112:0/765632238 conn(0x55e0e8fd9300 msgr2=0x55e0e9384880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 -- 172.21.15.112:0/765632238 shutdown_connections 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 -- 172.21.15.112:0/765632238 wait complete. 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 -- start start 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.497+0000 7fae82b98740 1 Processor -- start 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 0x55e0e939a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fc44d0 0x55e0e939aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e940ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae741ac150 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae741ad5c0 con 0x55e0e8fe51b0 2024-04-25T16:49:41.500 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae741ac820 con 0x55e0e8fc44d0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae82b98740 10 client.0 _ll_register_callbacks cb 0x7fae700123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 0x55e0e939a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 0x55e0e939a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40994/0 (socket says 172.21.15.112:40994) 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 -- 172.21.15.112:0/2932783465 learned_addr learned my addr 172.21.15.112:0/2932783465 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79282640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fc44d0 0x55e0e939aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae7a284640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e940ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 -- 172.21.15.112:0/2932783465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fc44d0 msgr2=0x55e0e939aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fc44d0 0x55e0e939aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 -- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 msgr2=0x55e0e940ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e940ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 -- 172.21.15.112:0/2932783465 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0e939b310 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae79a83640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 0x55e0e939a530 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009a60 tx=0x7fae6c031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3b571b9abcf00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.498+0000 7fae5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae5d7fa640 1 -- 172.21.15.112:0/2932783465 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c031860 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae5d7fa640 1 -- 172.21.15.112:0/2932783465 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7fae6c003970 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae640a7450 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae5d7fa640 1 -- 172.21.15.112:0/2932783465 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fae6c003b70 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 10 client.16335 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e0e940d090 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae5d7fa640 1 -- 172.21.15.112:0/2932783465 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae640a7450 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae5d7fa640 1 -- 172.21.15.112:0/2932783465 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7fae6c02f6f0 con 0x55e0e8fbc2f0 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 10 client.16335 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 20 client.16335 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8f791c0 0x55e0e940f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 4 client.16335 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:49:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0e92f2ee0 con 0x55e0e8f791c0 2024-04-25T16:49:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.499+0000 7fae79282640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8f791c0 0x55e0e940f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.500+0000 7fae5d7fa640 10 client.16335 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.500+0000 7fae79282640 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8f791c0 0x55e0e940f8d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a6d6c2f023a65a server_cookie=98bec028c37aef13 in_seq=0 out_seq=0 2024-04-25T16:49:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:41 smithi116 ceph-mon[32712]: pgmap v356: 97 pgs: 97 active+clean; 83 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-04-25T16:49:41.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:41 smithi112 ceph-mon[29122]: pgmap v356: 97 pgs: 97 active+clean; 83 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.1 MiB/s wr, 308 op/s 2024-04-25T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae5d7fa640 1 -- 172.21.15.112:0/2932783465 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0e92f2ee0 con 0x55e0e8f791c0 2024-04-25T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae5d7fa640 10 client.16335 handle_command_reply: tid=0 2024-04-25T16:49:41.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 client.16335 shutdown 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 msgr2=0x55e0e939a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 0x55e0e939a530 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009a60 tx=0x7fae6c031db0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 shutdown_connections 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55e0e8f791c0 0x55e0e940f8d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e0e8fc44d0 0x55e0e939aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e0e8fe51b0 0x55e0e940ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 --2- 172.21.15.112:0/2932783465 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e0e8fbc2f0 0x55e0e939a530 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 >> 172.21.15.112:0/2932783465 conn(0x55e0e8fd9300 msgr2=0x55e0e93852b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae5d7fa640 0 client.16335 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 shutdown_connections 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 1 -- 172.21.15.112:0/2932783465 wait complete. 2024-04-25T16:49:41.666 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.667+0000 7fae82b98740 20 client.16335 trim_cache size 0 max 16384 2024-04-25T16:49:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:49:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:49:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:49:41.667 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:49:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fae600afc50 msgr2=0x7fae600b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fae600afc50 0x7fae600b2160 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fae7000b590 tx=0x7fae7000baa0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 msgr2=0x7fae7419ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae7419ad10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fae6400e9f0 tx=0x7fae6400eeb0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 shutdown_connections 2024-04-25T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fae600afc50 0x7fae600b2160 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.669 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae7410a040 0x7fae7419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae74109ba0 0x7fae7419ad10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 --2- 172.21.15.112:0/1484892994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae741042f0 0x7fae7419a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 >> 172.21.15.112:0/1484892994 conn(0x7fae740792a0 msgr2=0x7fae74105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 shutdown_connections 2024-04-25T16:49:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:41.668+0000 7fae7bce7640 1 -- 172.21.15.112:0/1484892994 wait complete. 2024-04-25T16:49:41.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1183969 10.29 MB/sec execute 467 sec latency 119.019 ms 2024-04-25T16:49:42.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1186292 10.30 MB/sec execute 468 sec latency 18.830 ms 2024-04-25T16:49:43.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:43 smithi133 ceph-mon[32691]: pgmap v357: 97 pgs: 97 active+clean; 83 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-04-25T16:49:43.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:43 smithi116 ceph-mon[32712]: pgmap v357: 97 pgs: 97 active+clean; 83 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-04-25T16:49:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:43 smithi112 ceph-mon[29122]: pgmap v357: 97 pgs: 97 active+clean; 83 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 154 op/s 2024-04-25T16:49:43.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1187270 10.28 MB/sec execute 469 sec latency 25.053 ms 2024-04-25T16:49:44.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1189281 10.28 MB/sec execute 470 sec latency 28.711 ms 2024-04-25T16:49:45.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:45 smithi133 ceph-mon[32691]: pgmap v358: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-04-25T16:49:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:49:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:45 smithi116 ceph-mon[32712]: pgmap v358: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-04-25T16:49:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:49:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:45 smithi112 ceph-mon[29122]: pgmap v358: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 304 op/s 2024-04-25T16:49:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:49:45.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1190425 10.28 MB/sec execute 471 sec latency 18.982 ms 2024-04-25T16:49:46.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:46 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:46.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:46 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:46.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:46 smithi112 ceph-mon[29122]: pgmap v359: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-04-25T16:49:46.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:46 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:49:46.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:46 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:49:46.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:46 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:46.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1192337 10.27 MB/sec execute 472 sec latency 19.452 ms 2024-04-25T16:49:46.948 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:46 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:46 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:46 smithi133 ceph-mon[32691]: pgmap v359: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-04-25T16:49:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:46 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:49:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:46 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:49:46.949 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:46 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:46 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:46 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:46 smithi116 ceph-mon[32712]: pgmap v359: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-04-25T16:49:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:46 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:49:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:46 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:49:47.008 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:46 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:49:47.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1193890 10.27 MB/sec execute 473 sec latency 27.098 ms 2024-04-25T16:49:48.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1195823 10.27 MB/sec execute 474 sec latency 16.491 ms 2024-04-25T16:49:49.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:48 smithi116 ceph-mon[32712]: pgmap v360: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-04-25T16:49:49.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:48 smithi112 ceph-mon[29122]: pgmap v360: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-04-25T16:49:49.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:48 smithi133 ceph-mon[32691]: pgmap v360: 97 pgs: 97 active+clean; 73 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 225 op/s 2024-04-25T16:49:49.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1197238 10.27 MB/sec execute 475 sec latency 64.848 ms 2024-04-25T16:49:50.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1198791 10.26 MB/sec execute 476 sec latency 50.511 ms 2024-04-25T16:49:51.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:50 smithi116 ceph-mon[32712]: pgmap v361: 97 pgs: 97 active+clean; 79 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-04-25T16:49:51.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:50 smithi112 ceph-mon[29122]: pgmap v361: 97 pgs: 97 active+clean; 79 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-04-25T16:49:51.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:50 smithi133 ceph-mon[32691]: pgmap v361: 97 pgs: 97 active+clean; 79 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.3 MiB/s wr, 304 op/s 2024-04-25T16:49:51.893 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1200700 10.27 MB/sec execute 477 sec latency 28.361 ms 2024-04-25T16:49:52.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1202248 10.26 MB/sec execute 478 sec latency 17.982 ms 2024-04-25T16:49:53.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:52 smithi116 ceph-mon[32712]: pgmap v362: 97 pgs: 97 active+clean; 79 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-04-25T16:49:53.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:52 smithi112 ceph-mon[29122]: pgmap v362: 97 pgs: 97 active+clean; 79 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-04-25T16:49:53.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:52 smithi133 ceph-mon[32691]: pgmap v362: 97 pgs: 97 active+clean; 79 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 228 op/s 2024-04-25T16:49:53.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1204258 10.26 MB/sec execute 479 sec latency 15.551 ms 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 -- 172.21.15.112:0/549985918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 msgr2=0x7f31e0102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 --2- 172.21.15.112:0/549985918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0102610 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009900 tx=0x7f31d002a410 comp rx=0 tx=0).stop 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 -- 172.21.15.112:0/549985918 shutdown_connections 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 --2- 172.21.15.112:0/549985918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e01031c0 0x7f31e010f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 --2- 172.21.15.112:0/549985918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31e0108f00 0x7f31e0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 --2- 172.21.15.112:0/549985918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0102610 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 -- 172.21.15.112:0/549985918 >> 172.21.15.112:0/549985918 conn(0x7f31e00781b0 msgr2=0x7f31e00785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 -- 172.21.15.112:0/549985918 shutdown_connections 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.771+0000 7f31e8051640 1 -- 172.21.15.112:0/549985918 wait complete. 2024-04-25T16:49:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 Processor -- start 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 -- start start 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31e01031c0 0x7f31e0199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e0108f00 0x7f31e019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0112c60 con 0x7f31e0108a60 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0112ae0 con 0x7f31e01031c0 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e8051640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31e0112de0 con 0x7f31e0108f00 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36382/0 (socket says 172.21.15.112:36382) 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e55ec640 1 -- 172.21.15.112:0/2185098548 learned_addr learned my addr 172.21.15.112:0/2185098548 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.772+0000 7f31e5ded640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31e01031c0 0x7f31e0199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e65ee640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e0108f00 0x7f31e019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e55ec640 1 -- 172.21.15.112:0/2185098548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e0108f00 msgr2=0x7f31e019dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e55ec640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e0108f00 0x7f31e019dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e55ec640 1 -- 172.21.15.112:0/2185098548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31e01031c0 msgr2=0x7f31e0199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e55ec640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31e01031c0 0x7f31e0199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e55ec640 1 -- 172.21.15.112:0/2185098548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e019e4e0 con 0x7f31e0108a60 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e65ee640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e0108f00 0x7f31e019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e55ec640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0199be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00e9f0 tx=0x7f31dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31d6ffd640 1 -- 172.21.15.112:0/2185098548 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00ccc0 con 0x7f31e0108a60 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e019e7d0 con 0x7f31e0108a60 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31d6ffd640 1 -- 172.21.15.112:0/2185098548 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc0035f0 con 0x7f31e0108a60 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e01a6150 con 0x7f31e0108a60 2024-04-25T16:49:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.773+0000 7f31d6ffd640 1 -- 172.21.15.112:0/2185098548 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00ce60 con 0x7f31e0108a60 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.775+0000 7f31d6ffd640 1 -- 172.21.15.112:0/2185098548 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f31dc003aa0 con 0x7f31e0108a60 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 Processor -- start 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 -- start start 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8678af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8679030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8679570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fdf85df460 con 0x55fdf82b84d0 2024-04-25T16:49:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fdf8599590 con 0x55fdf82d91b0 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31eefe4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fdf85db420 con 0x55fdf82b02f0 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8679570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e65ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8678af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e5ded640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8679030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e65ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8678af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:55144/0 (socket says 172.21.15.112:55144) 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8679570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36398/0 (socket says 172.21.15.112:36398) 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e5ded640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8679030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:49920/0 (socket says 172.21.15.112:49920) 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31e65ee640 1 -- 172.21.15.112:0/1578797959 learned_addr learned my addr 172.21.15.112:0/1578797959 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.776+0000 7f31d6ffd640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f31cc0b1e60 0x7f31cc0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e65ee640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f31cc0b1e60 0x7f31cc0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31d6ffd640 1 -- 172.21.15.112:0/2185098548 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f31dc0df410 con 0x7f31e0108a60 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e55ec640 1 -- 172.21.15.112:0/1578797959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 msgr2=0x55fdf8679030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8679030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e55ec640 1 -- 172.21.15.112:0/1578797959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 msgr2=0x55fdf8678af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8678af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e55ec640 1 -- 172.21.15.112:0/1578797959 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdf85a60d0 con 0x55fdf82b84d0 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8679030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e65ee640 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8678af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8679570 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0dcb10 tx=0x7f31dc003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36507faae49c30b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31e65ee640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f31cc0b1e60 0x7f31cc0b4370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55fdf8675c60 tx=0x7f31d8004ab0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31cbfff640 1 -- 172.21.15.112:0/1578797959 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0a7070 con 0x55fdf82b84d0 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31cbfff640 1 -- 172.21.15.112:0/1578797959 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc0a7c60 con 0x55fdf82b84d0 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.777+0000 7f31cbfff640 1 -- 172.21.15.112:0/1578797959 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0aff30 con 0x55fdf82b84d0 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 -- 172.21.15.112:0/1578797959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 msgr2=0x55fdf8679570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8679570 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0dcb10 tx=0x7f31dc003a10 comp rx=0 tx=0).stop 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 -- 172.21.15.112:0/1578797959 shutdown_connections 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8679030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8679570 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1578797959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8678af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 -- 172.21.15.112:0/1578797959 >> 172.21.15.112:0/1578797959 conn(0x55fdf82cd300 msgr2=0x55fdf86752a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 -- 172.21.15.112:0/1578797959 shutdown_connections 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 -- 172.21.15.112:0/1578797959 wait complete. 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.778+0000 7f31eefe4740 1 -- start start 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 Processor -- start 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8684b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8685090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8683040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55fdf8685bc0 con 0x55fdf82b84d0 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fdf8566b60 con 0x55fdf82d91b0 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fdf8685d40 con 0x55fdf82b02f0 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31eefe4740 10 client.0 _ll_register_callbacks cb 0x55fdf858b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8685090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8685090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36408/0 (socket says 172.21.15.112:36408) 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e65ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8683040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 -- 172.21.15.112:0/1266284970 learned_addr learned my addr 172.21.15.112:0/1266284970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8684b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 -- 172.21.15.112:0/1266284970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 msgr2=0x55fdf8684b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8684b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 -- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 msgr2=0x55fdf8683040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8683040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.779+0000 7f31e55ec640 1 -- 172.21.15.112:0/1266284970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdf82f1210 con 0x55fdf82b84d0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8685090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0a7260 tx=0x7f31dc0a2df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a425a120ea653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31c97fa640 1 -- 172.21.15.112:0/1266284970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0a7070 con 0x55fdf82b84d0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31dc0a7c60 con 0x55fdf82b84d0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31c97fa640 1 -- 172.21.15.112:0/1266284970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f31dc0aff30 con 0x55fdf82b84d0 2024-04-25T16:49:54.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31c97fa640 1 -- 172.21.15.112:0/1266284970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31dc0b03c0 con 0x55fdf82b84d0 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 10 client.16347 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fdf83bd620 con 0x55fdf82b84d0 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31c97fa640 1 -- 172.21.15.112:0/1266284970 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc0a7c60 con 0x55fdf82b84d0 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31c97fa640 1 -- 172.21.15.112:0/1266284970 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f31dc0a2ed0 con 0x55fdf82b84d0 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 10 client.16347 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 20 client.16347 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf8297d80 0x55fdf8695600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 4 client.16347 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.780+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdf85ed070 con 0x55fdf8297d80 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.781+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf8297d80 0x55fdf8695600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.781+0000 7f31c97fa640 10 client.16347 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.781+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf8297d80 0x55fdf8695600 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6011aae74f91468f server_cookie=26088ad3dd8305f2 in_seq=0 out_seq=0 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31c97fa640 1 -- 172.21.15.112:0/1266284970 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fdf85ed070 con 0x55fdf8297d80 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31c97fa640 10 client.16347 handle_command_reply: tid=0 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 client.16347 shutdown 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 msgr2=0x55fdf8685090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8685090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f31dc0a7260 tx=0x7f31dc0a2df0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 shutdown_connections 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82d91b0 0x55fdf8683040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8685090 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf8297d80 0x55fdf8695600 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1266284970 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82b02f0 0x55fdf8684b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 >> 172.21.15.112:0/1266284970 conn(0x55fdf82cd300 msgr2=0x55fdf867a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31c97fa640 0 client.16347 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 shutdown_connections 2024-04-25T16:49:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 1 -- 172.21.15.112:0/1266284970 wait complete. 2024-04-25T16:49:54.789 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.782+0000 7f31eefe4740 20 client.16347 trim_cache size 0 max 16384 2024-04-25T16:49:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.798+0000 7f31eefe4740 1 Processor -- start 2024-04-25T16:49:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.798+0000 7f31eefe4740 1 -- start start 2024-04-25T16:49:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b02f0 0x55fdf8679980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8684e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf867c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31eefe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01b0300 con 0x55fdf82b84d0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31eefe4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01adbd0 con 0x55fdf82b02f0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31eefe4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01aba40 con 0x55fdf82d91b0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8684e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8684e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36416/0 (socket says 172.21.15.112:36416) 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 -- 172.21.15.112:0/4133280667 learned_addr learned my addr 172.21.15.112:0/4133280667 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e5ded640 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b02f0 0x55fdf8679980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e65ee640 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf867c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 -- 172.21.15.112:0/4133280667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 msgr2=0x55fdf867c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf867c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 -- 172.21.15.112:0/4133280667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b02f0 msgr2=0x55fdf8679980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b02f0 0x55fdf8679980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 -- 172.21.15.112:0/4133280667 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdf8676380 con 0x55fdf82b84d0 2024-04-25T16:49:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e65ee640 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf867c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31e55ec640 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8684e70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c6a0 tx=0x7f31dc0df3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7706cfe5e45c8cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31cb7fe640 1 -- 172.21.15.112:0/4133280667 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0e4070 con 0x55fdf82b84d0 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31cb7fe640 1 -- 172.21.15.112:0/4133280667 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc0a7070 con 0x55fdf82b84d0 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.799+0000 7f31cb7fe640 1 -- 172.21.15.112:0/4133280667 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc0aff30 con 0x55fdf82b84d0 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 -- 172.21.15.112:0/4133280667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 msgr2=0x55fdf8684e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8684e70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00c6a0 tx=0x7f31dc0df3a0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 -- 172.21.15.112:0/4133280667 shutdown_connections 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf867c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b84d0 0x55fdf8684e70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 --2- 172.21.15.112:0/4133280667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b02f0 0x55fdf8679980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 -- 172.21.15.112:0/4133280667 >> 172.21.15.112:0/4133280667 conn(0x55fdf82cd300 msgr2=0x55fdf8684760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 -- 172.21.15.112:0/4133280667 shutdown_connections 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 -- 172.21.15.112:0/4133280667 wait complete. 2024-04-25T16:49:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.800+0000 7f31eefe4740 1 -- start start 2024-04-25T16:49:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 Processor -- start 2024-04-25T16:49:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 0x55fdf8695990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b84d0 0x55fdf8695ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf86fb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01ac090 con 0x55fdf82b02f0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01ad500 con 0x55fdf82b84d0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31e01ac760 con 0x55fdf82d91b0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31eefe4740 10 client.0 _ll_register_callbacks cb 0x7f31d80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 0x55fdf8695990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 0x55fdf8695990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36432/0 (socket says 172.21.15.112:36432) 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 -- 172.21.15.112:0/1812950528 learned_addr learned my addr 172.21.15.112:0/1812950528 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b84d0 0x55fdf8695ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e65ee640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf86fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 -- 172.21.15.112:0/1812950528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 msgr2=0x55fdf86fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf86fb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 -- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b84d0 msgr2=0x55fdf8695ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:49:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b84d0 0x55fdf8695ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.801+0000 7f31e5ded640 1 -- 172.21.15.112:0/1812950528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdf86fbc60 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31e5ded640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 0x55fdf8695990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009230 tx=0x7f31d0005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e60d79fb6564999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31c97fa640 1 -- 172.21.15.112:0/1812950528 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0035040 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31dc0a7070 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31c97fa640 1 -- 172.21.15.112:0/1812950528 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8600+0+0 (secure 0 0 0) 0x7f31d0005040 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31c97fa640 1 -- 172.21.15.112:0/1812950528 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f31d0055050 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 10 client.16353 fetch_fsmap learned FSMap version 34 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fdf86fcdf0 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31c97fa640 1 -- 172.21.15.112:0/1812950528 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31dc0a7070 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31c97fa640 1 -- 172.21.15.112:0/1812950528 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7f31d0046320 con 0x55fdf82b02f0 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 10 client.16353 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:49:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 20 client.16353 populate_metadata read hostname 'smithi112' 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf826d1c0 0x55fdf86ff580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 4 client.16353 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.802+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fdf85e6ee0 con 0x55fdf826d1c0 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.803+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf826d1c0 0x55fdf86ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.803+0000 7f31c97fa640 10 client.16353 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.803+0000 7f31e55ec640 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf826d1c0 0x55fdf86ff580 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbff04ec974025a9 server_cookie=4528efcff08e0faa in_seq=0 out_seq=0 2024-04-25T16:49:54.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1205579 10.25 MB/sec execute 480 sec latency 111.304 ms 2024-04-25T16:49:54.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31c97fa640 1 -- 172.21.15.112:0/1812950528 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fdf85e6ee0 con 0x55fdf826d1c0 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31c97fa640 10 client.16353 handle_command_reply: tid=0 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 client.16353 shutdown 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 msgr2=0x55fdf8695990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 0x55fdf8695990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f31d0009230 tx=0x7f31d0005ea0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 shutdown_connections 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55fdf826d1c0 0x55fdf86ff580 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fdf82d91b0 0x55fdf86fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31c97fa640 0 client.16353 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fdf82b84d0 0x55fdf8695ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 --2- 172.21.15.112:0/1812950528 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55fdf82b02f0 0x55fdf8695990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 >> 172.21.15.112:0/1812950528 conn(0x55fdf82cd300 msgr2=0x55fdf867adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 shutdown_connections 2024-04-25T16:49:54.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 1 -- 172.21.15.112:0/1812950528 wait complete. 2024-04-25T16:49:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.978+0000 7f31eefe4740 20 client.16353 trim_cache size 0 max 16384 2024-04-25T16:49:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:49:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:49:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:49:54.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f31cc0b1e60 msgr2=0x7f31cc0b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f31cc0b1e60 0x7f31cc0b4370 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55fdf8675c60 tx=0x7f31d8004ab0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 msgr2=0x7f31e0199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0199be0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00e9f0 tx=0x7f31dc00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 shutdown_connections 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f31cc0b1e60 0x7f31cc0b4370 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31e0108f00 0x7f31e019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f31e0108a60 0x7f31e0199be0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 --2- 172.21.15.112:0/2185098548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31e01031c0 0x7f31e0199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:49:54.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 >> 172.21.15.112:0/2185098548 conn(0x7f31e00781b0 msgr2=0x7f31e0104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 shutdown_connections 2024-04-25T16:49:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:49:54.979+0000 7f31e8051640 1 -- 172.21.15.112:0/2185098548 wait complete. 2024-04-25T16:49:55.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:54 smithi116 ceph-mon[32712]: pgmap v363: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-04-25T16:49:55.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:54 smithi112 ceph-mon[29122]: pgmap v363: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-04-25T16:49:55.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:54 smithi133 ceph-mon[32691]: pgmap v363: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 310 op/s 2024-04-25T16:49:55.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1207638 10.26 MB/sec execute 481 sec latency 125.295 ms 2024-04-25T16:49:56.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1208998 10.25 MB/sec execute 482 sec latency 16.339 ms 2024-04-25T16:49:57.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:56 smithi116 ceph-mon[32712]: pgmap v364: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 160 op/s 2024-04-25T16:49:57.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:56 smithi112 ceph-mon[29122]: pgmap v364: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 160 op/s 2024-04-25T16:49:57.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:56 smithi133 ceph-mon[32691]: pgmap v364: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 160 op/s 2024-04-25T16:49:57.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1211214 10.26 MB/sec execute 483 sec latency 17.205 ms 2024-04-25T16:49:58.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1212671 10.25 MB/sec execute 484 sec latency 17.750 ms 2024-04-25T16:49:59.001 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:49:58 smithi133 ceph-mon[32691]: pgmap v365: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 160 op/s 2024-04-25T16:49:59.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:49:58 smithi116 ceph-mon[32712]: pgmap v365: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 160 op/s 2024-04-25T16:49:59.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:49:58 smithi112 ceph-mon[29122]: pgmap v365: 97 pgs: 97 active+clean; 85 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 160 op/s 2024-04-25T16:49:59.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1214746 10.25 MB/sec execute 485 sec latency 69.710 ms 2024-04-25T16:50:00.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1215678 10.24 MB/sec execute 486 sec latency 32.234 ms 2024-04-25T16:50:01.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:00 smithi116 ceph-mon[32712]: pgmap v366: 97 pgs: 97 active+clean; 71 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 322 op/s 2024-04-25T16:50:01.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:00 smithi116 ceph-mon[32712]: overall HEALTH_OK 2024-04-25T16:50:01.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:00 smithi112 ceph-mon[29122]: pgmap v366: 97 pgs: 97 active+clean; 71 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 322 op/s 2024-04-25T16:50:01.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:00 smithi112 ceph-mon[29122]: overall HEALTH_OK 2024-04-25T16:50:01.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:00 smithi133 ceph-mon[32691]: pgmap v366: 97 pgs: 97 active+clean; 71 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.8 MiB/s wr, 322 op/s 2024-04-25T16:50:01.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:00 smithi133 ceph-mon[32691]: overall HEALTH_OK 2024-04-25T16:50:01.894 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1217212 10.23 MB/sec execute 487 sec latency 29.674 ms 2024-04-25T16:50:02.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1218665 10.23 MB/sec execute 488 sec latency 15.028 ms 2024-04-25T16:50:03.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:02 smithi116 ceph-mon[32712]: pgmap v367: 97 pgs: 97 active+clean; 71 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-25T16:50:03.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:02 smithi112 ceph-mon[29122]: pgmap v367: 97 pgs: 97 active+clean; 71 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-25T16:50:03.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:02 smithi133 ceph-mon[32691]: pgmap v367: 97 pgs: 97 active+clean; 71 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-04-25T16:50:03.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1220617 10.23 MB/sec execute 489 sec latency 45.274 ms 2024-04-25T16:50:04.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1222074 10.23 MB/sec execute 490 sec latency 106.729 ms 2024-04-25T16:50:05.007 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:04 smithi116 ceph-mon[32712]: pgmap v368: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-04-25T16:50:05.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:04 smithi112 ceph-mon[29122]: pgmap v368: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-04-25T16:50:05.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:04 smithi133 ceph-mon[32691]: pgmap v368: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.8 MiB/s wr, 342 op/s 2024-04-25T16:50:05.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1223412 10.22 MB/sec execute 491 sec latency 52.675 ms 2024-04-25T16:50:06.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1224131 10.20 MB/sec execute 492 sec latency 74.069 ms 2024-04-25T16:50:07.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:06 smithi112 ceph-mon[29122]: pgmap v369: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-04-25T16:50:07.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:06 smithi112 ceph-mon[29122]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T16:50:07.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:06 smithi133 ceph-mon[32691]: pgmap v369: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-04-25T16:50:07.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:06 smithi133 ceph-mon[32691]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T16:50:07.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:06 smithi116 ceph-mon[32712]: pgmap v369: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-04-25T16:50:07.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:06 smithi116 ceph-mon[32712]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T16:50:07.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1224774 10.19 MB/sec execute 493 sec latency 57.010 ms 2024-04-25T16:50:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 -- 172.21.15.112:0/1516822969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 msgr2=0x7feecc105140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 --2- 172.21.15.112:0/1516822969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc105140 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7feebc009900 tx=0x7feebc02a480 comp rx=0 tx=0).stop 2024-04-25T16:50:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 -- 172.21.15.112:0/1516822969 shutdown_connections 2024-04-25T16:50:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 --2- 172.21.15.112:0/1516822969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 --2- 172.21.15.112:0/1516822969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feecc10e0f0 0x7feecc1105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 --2- 172.21.15.112:0/1516822969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc105140 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.120+0000 7feed13b2640 1 -- 172.21.15.112:0/1516822969 >> 172.21.15.112:0/1516822969 conn(0x7feecc078080 msgr2=0x7feecc0784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.121+0000 7feed13b2640 1 -- 172.21.15.112:0/1516822969 shutdown_connections 2024-04-25T16:50:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.121+0000 7feed13b2640 1 -- 172.21.15.112:0/1516822969 wait complete. 2024-04-25T16:50:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.122+0000 7feed13b2640 1 Processor -- start 2024-04-25T16:50:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.122+0000 7feed13b2640 1 -- start start 2024-04-25T16:50:08.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.122+0000 7feed13b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feed13b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feecaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feed13b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feecc10e0f0 0x7feecc19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feecaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35720/0 (socket says 172.21.15.112:35720) 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feed13b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feecc113d50 con 0x7feecc104d30 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc19acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:40642/0 (socket says 172.21.15.112:40642) 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 -- 172.21.15.112:0/3242232206 learned_addr learned my addr 172.21.15.112:0/3242232206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feecc113bd0 con 0x7feecc10e0f0 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feecc113ed0 con 0x7feecc10b790 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feecb7fe640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feecc10e0f0 0x7feecc19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 -- 172.21.15.112:0/3242232206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feecc10e0f0 msgr2=0x7feecc19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feecc10e0f0 0x7feecc19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 -- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 msgr2=0x7feecc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.123+0000 7feeca7fc640 1 -- 172.21.15.112:0/3242232206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feecc19f5b0 con 0x7feecc10b790 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.124+0000 7feeca7fc640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc19acb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feec000ea60 tx=0x7feec000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:08.123 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.124+0000 7feeb3fff640 1 -- 172.21.15.112:0/3242232206 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec000cde0 con 0x7feecc10b790 2024-04-25T16:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.124+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feecc19f8a0 con 0x7feecc10b790 2024-04-25T16:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.124+0000 7feeb3fff640 1 -- 172.21.15.112:0/3242232206 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00035f0 con 0x7feecc10b790 2024-04-25T16:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.124+0000 7feeb3fff640 1 -- 172.21.15.112:0/3242232206 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0010690 con 0x7feecc10b790 2024-04-25T16:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.124+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feecc1a7220 con 0x7feecc10b790 2024-04-25T16:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.127+0000 7feed8263740 1 Processor -- start 2024-04-25T16:50:08.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.127+0000 7feeb3fff640 1 -- 172.21.15.112:0/3242232206 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7feec0010830 con 0x7feecc10b790 2024-04-25T16:50:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.127+0000 7feed8263740 1 -- start start 2024-04-25T16:50:08.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeb3fff640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feeb40afc50 0x7feeb40b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feed8263740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feecb7fe640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feeb40afc50 0x7feeb40b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feed8263740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb93b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feed8263740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb939800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feed8263740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b7cb892460 con 0x55b7cb5632f0 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feed8263740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b7cb84c590 con 0x55b7cb58c1b0 2024-04-25T16:50:08.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feed8263740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b7cb88e420 con 0x55b7cb56b4d0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeb3fff640 1 -- 172.21.15.112:0/3242232206 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7feec00df410 con 0x7feecc10b790 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feecaffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb93b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feecaffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:33252/0 (socket says 172.21.15.112:33252) 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feecaffd640 1 -- 172.21.15.112:0/224233774 learned_addr learned my addr 172.21.15.112:0/224233774 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feecb7fe640 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb939800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeca7fc640 1 -- 172.21.15.112:0/224233774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 msgr2=0x55b7cb939800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeca7fc640 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb939800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeca7fc640 1 -- 172.21.15.112:0/224233774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 msgr2=0x55b7cb93ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeca7fc640 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.128+0000 7feeca7fc640 1 -- 172.21.15.112:0/224233774 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7cb8590d0 con 0x55b7cb5632f0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.129+0000 7feeca7fc640 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb93b0a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7feec00dcae0 tx=0x7feec00a6f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f6a0323b448990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.129+0000 7feecb7fe640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feeb40afc50 0x7feeb40b2160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec4012a70 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.129+0000 7feeb17fa640 1 -- 172.21.15.112:0/224233774 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00a7c30 con 0x55b7cb5632f0 2024-04-25T16:50:08.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.129+0000 7feeb17fa640 1 -- 172.21.15.112:0/224233774 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00a7530 con 0x55b7cb5632f0 2024-04-25T16:50:08.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.129+0000 7feeb17fa640 1 -- 172.21.15.112:0/224233774 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00a37c0 con 0x55b7cb5632f0 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 -- 172.21.15.112:0/224233774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 msgr2=0x55b7cb93b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb93b0a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7feec00dcae0 tx=0x7feec00a6f40 comp rx=0 tx=0).stop 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 -- 172.21.15.112:0/224233774 shutdown_connections 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb939800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb93b0a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 --2- 172.21.15.112:0/224233774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 -- 172.21.15.112:0/224233774 >> 172.21.15.112:0/224233774 conn(0x55b7cb580300 msgr2=0x55b7cb927c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 -- 172.21.15.112:0/224233774 shutdown_connections 2024-04-25T16:50:08.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.130+0000 7feed8263740 1 -- 172.21.15.112:0/224233774 wait complete. 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 -- start start 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 Processor -- start 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb944a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb944f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b7cb93a170 con 0x55b7cb5632f0 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b7cb819b60 con 0x55b7cb58c1b0 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b7cb93a2f0 con 0x55b7cb56b4d0 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feecaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb944a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feecaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb944a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35740/0 (socket says 172.21.15.112:35740) 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feecaffd640 1 -- 172.21.15.112:0/2635840921 learned_addr learned my addr 172.21.15.112:0/2635840921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feeca7fc640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb944f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.131+0000 7feed8263740 10 client.0 _ll_register_callbacks cb 0x55b7cb83e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecb7fe640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecaffd640 1 -- 172.21.15.112:0/2635840921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 msgr2=0x55b7cb944f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecaffd640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb944f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecaffd640 1 -- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 msgr2=0x55b7cb93edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecaffd640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecaffd640 1 -- 172.21.15.112:0/2635840921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7cb5a4210 con 0x55b7cb5632f0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feecaffd640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb944a00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7feebc009230 tx=0x7feebc032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b510eeb8011824e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feeaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feeaeff5640 1 -- 172.21.15.112:0/2635840921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc035040 con 0x55b7cb5632f0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feeaeff5640 1 -- 172.21.15.112:0/2635840921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7feebc004040 con 0x55b7cb5632f0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.132+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec00a7530 con 0x55b7cb5632f0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.133+0000 7feeaeff5640 1 -- 172.21.15.112:0/2635840921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feebc0318a0 con 0x55b7cb5632f0 2024-04-25T16:50:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.133+0000 7feed8263740 10 client.16365 fetch_fsmap learned FSMap version 34 2024-04-25T16:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.133+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b7cb670620 con 0x55b7cb5632f0 2024-04-25T16:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feeaeff5640 1 -- 172.21.15.112:0/2635840921 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00a7530 con 0x55b7cb5632f0 2024-04-25T16:50:08.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feeaeff5640 1 -- 172.21.15.112:0/2635840921 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7feebc0047b0 con 0x55b7cb5632f0 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feed8263740 10 client.16365 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feed8263740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feed8263740 20 client.16365 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feed8263740 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb54ad80 0x55b7cb950e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feed8263740 4 client.16365 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.134+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7cb8a0070 con 0x55b7cb54ad80 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.135+0000 7feeca7fc640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb54ad80 0x55b7cb950e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.136+0000 7feeca7fc640 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb54ad80 0x55b7cb950e00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61824cc75cf5b2c9 server_cookie=43452ca7d7dd400 in_seq=0 out_seq=0 2024-04-25T16:50:08.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.136+0000 7feeaeff5640 10 client.16365 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feeaeff5640 1 -- 172.21.15.112:0/2635840921 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b7cb8a0070 con 0x55b7cb54ad80 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feeaeff5640 10 client.16365 handle_command_reply: tid=0 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 client.16365 shutdown 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 msgr2=0x55b7cb944a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb944a00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7feebc009230 tx=0x7feebc032d50 comp rx=0 tx=0).stop 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 shutdown_connections 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb54ad80 0x55b7cb950e00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb56b4d0 0x55b7cb944f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb58c1b0 0x55b7cb93edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 --2- 172.21.15.112:0/2635840921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb5632f0 0x55b7cb944a00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.138+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 >> 172.21.15.112:0/2635840921 conn(0x55b7cb580300 msgr2=0x55b7cb927c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.139+0000 7feeaeff5640 0 client.16365 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.139+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 shutdown_connections 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.139+0000 7feed8263740 1 -- 172.21.15.112:0/2635840921 wait complete. 2024-04-25T16:50:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.139+0000 7feed8263740 20 client.16365 trim_cache size 0 max 16384 2024-04-25T16:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 Processor -- start 2024-04-25T16:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 -- start start 2024-04-25T16:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb5632f0 0x55b7cb93d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 0x55b7cb93dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.159 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1b0450 con 0x55b7cb56b4d0 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1add20 con 0x55b7cb5632f0 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feed8263740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1abb90 con 0x55b7cb58c1b0 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 0x55b7cb93dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feeca7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 0x55b7cb93dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35750/0 (socket says 172.21.15.112:35750) 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feeca7fc640 1 -- 172.21.15.112:0/1837918468 learned_addr learned my addr 172.21.15.112:0/1837918468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.160+0000 7feecaffd640 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb5632f0 0x55b7cb93d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feecb7fe640 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeca7fc640 1 -- 172.21.15.112:0/1837918468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb58c1b0 msgr2=0x55b7cb9565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeca7fc640 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeca7fc640 1 -- 172.21.15.112:0/1837918468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb5632f0 msgr2=0x55b7cb93d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeca7fc640 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb5632f0 0x55b7cb93d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeca7fc640 1 -- 172.21.15.112:0/1837918468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7cb93a500 con 0x55b7cb56b4d0 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeca7fc640 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 0x55b7cb93dc00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feec0009920 tx=0x7feec00a6fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f424dfb57139519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeb0ff9640 1 -- 172.21.15.112:0/1837918468 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ab8e0 con 0x55b7cb56b4d0 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeb0ff9640 1 -- 172.21.15.112:0/1837918468 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00b0390 con 0x55b7cb56b4d0 2024-04-25T16:50:08.161 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.161+0000 7feeb0ff9640 1 -- 172.21.15.112:0/1837918468 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0004260 con 0x55b7cb56b4d0 2024-04-25T16:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 -- 172.21.15.112:0/1837918468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 msgr2=0x55b7cb93dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 0x55b7cb93dc00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feec0009920 tx=0x7feec00a6fb0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 -- 172.21.15.112:0/1837918468 shutdown_connections 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb56b4d0 0x55b7cb93dc00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 --2- 172.21.15.112:0/1837918468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb5632f0 0x55b7cb93d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 -- 172.21.15.112:0/1837918468 >> 172.21.15.112:0/1837918468 conn(0x55b7cb580300 msgr2=0x55b7cb938390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 -- 172.21.15.112:0/1837918468 shutdown_connections 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.162+0000 7feed8263740 1 -- 172.21.15.112:0/1837918468 wait complete. 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 -- start start 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 Processor -- start 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb5632f0 0x55b7cb94abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb56b4d0 0x55b7cb94b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1ac1e0 con 0x55b7cb58c1b0 2024-04-25T16:50:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feecaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb5632f0 0x55b7cb94abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feecb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:35762/0 (socket says 172.21.15.112:35762) 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feecb7fe640 1 -- 172.21.15.112:0/4111413068 learned_addr learned my addr 172.21.15.112:0/4111413068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feecaffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb5632f0 0x55b7cb94abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:40676/0 (socket says 172.21.15.112:40676) 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.163+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1ad650 con 0x55b7cb56b4d0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feecc1ac8b0 con 0x55b7cb5632f0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feeca7fc640 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb56b4d0 0x55b7cb94b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feed8263740 10 client.0 _ll_register_callbacks cb 0x7feebc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feecb7fe640 1 -- 172.21.15.112:0/4111413068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb5632f0 msgr2=0x55b7cb94abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feecb7fe640 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb5632f0 0x55b7cb94abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feecb7fe640 1 -- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb56b4d0 msgr2=0x55b7cb94b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feecb7fe640 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb56b4d0 0x55b7cb94b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feecb7fe640 1 -- 172.21.15.112:0/4111413068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7cb92b450 con 0x55b7cb58c1b0 2024-04-25T16:50:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feecb7fe640 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9480f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7feec400ea90 tx=0x7feec401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f7d62a225f835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feeaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feeaeff5640 1 -- 172.21.15.112:0/4111413068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec402e060 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feeaeff5640 1 -- 172.21.15.112:0/4111413068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8644+0+0 (secure 0 0 0) 0x7feec4029480 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.164+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec00b0390 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feeaeff5640 1 -- 172.21.15.112:0/4111413068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feec4029680 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 10 client.16371 fetch_fsmap learned FSMap version 34 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b7cb9b89a0 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feeaeff5640 1 -- 172.21.15.112:0/4111413068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00b0390 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feeaeff5640 1 -- 172.21.15.112:0/4111413068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 34) ==== 8338+0+0 (secure 0 0 0) 0x7feec401e070 con 0x55b7cb58c1b0 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 10 client.16371 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 20 client.16371 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb5201c0 0x55b7cb9bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 4 client.16371 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:50:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b7cb899ee0 con 0x55b7cb5201c0 2024-04-25T16:50:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.165+0000 7feecaffd640 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb5201c0 0x55b7cb9bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.167+0000 7feeaeff5640 10 client.16371 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.167+0000 7feecaffd640 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb5201c0 0x55b7cb9bb170 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66cf5ad7d3a9b8f server_cookie=d0d9961a36b8183b in_seq=0 out_seq=0 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feeaeff5640 1 -- 172.21.15.112:0/4111413068 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b7cb899ee0 con 0x55b7cb5201c0 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feeaeff5640 10 client.16371 handle_command_reply: tid=0 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 client.16371 shutdown 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 msgr2=0x55b7cb9480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9480f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7feec400ea90 tx=0x7feec401a000 comp rx=0 tx=0).stop 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 shutdown_connections 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b7cb58c1b0 0x55b7cb9480f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7cb56b4d0 0x55b7cb94b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b7cb5201c0 0x55b7cb9bb170 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 --2- 172.21.15.112:0/4111413068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b7cb5632f0 0x55b7cb94abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.359 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feeaeff5640 0 client.16371 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 >> 172.21.15.112:0/4111413068 conn(0x55b7cb580300 msgr2=0x55b7cb938390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 shutdown_connections 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 1 -- 172.21.15.112:0/4111413068 wait complete. 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.360+0000 7feed8263740 20 client.16371 trim_cache size 0 max 16384 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:50:08.360 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:50:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feeb40afc50 msgr2=0x7feeb40b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feeb40afc50 0x7feeb40b2160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feec4005fd0 tx=0x7feec4012a70 comp rx=0 tx=0).stop 2024-04-25T16:50:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 msgr2=0x7feecc19acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:08.361 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc19acb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feec000ea60 tx=0x7feec000ef20 comp rx=0 tx=0).stop 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 shutdown_connections 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7feeb40afc50 0x7feeb40b2160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feecc10b790 0x7feecc19acb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feecc10e0f0 0x7feecc19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 --2- 172.21.15.112:0/3242232206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feecc104d30 0x7feecc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 >> 172.21.15.112:0/3242232206 conn(0x7feecc078080 msgr2=0x7feecc1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 shutdown_connections 2024-04-25T16:50:08.362 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:08.361+0000 7feed13b2640 1 -- 172.21.15.112:0/3242232206 wait complete. 2024-04-25T16:50:08.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1225601 10.18 MB/sec execute 494 sec latency 95.521 ms 2024-04-25T16:50:09.092 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:08 smithi116 ceph-mon[32712]: pgmap v371: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-04-25T16:50:09.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:08 smithi112 ceph-mon[29122]: pgmap v371: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-04-25T16:50:09.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:08 smithi133 ceph-mon[32691]: pgmap v371: 97 pgs: 97 active+clean; 77 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 312 op/s 2024-04-25T16:50:09.507 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:50:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setuser ceph since I am not root 2024-04-25T16:50:09.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:50:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setgroup ceph since I am not root 2024-04-25T16:50:09.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:50:09 smithi116 ceph-mds[62441]: -- 172.21.15.116:0/172565050 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c0f0adfa40 con 0x55c0f0b27180 2024-04-25T16:50:09.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:50:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: starting mds.k at 2024-04-25T16:50:09.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1226969 10.17 MB/sec execute 495 sec latency 102.835 ms 2024-04-25T16:50:10.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:09 smithi112 ceph-mon[29122]: daemon mds.k restarted 2024-04-25T16:50:10.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:09 smithi133 ceph-mon[32691]: daemon mds.k restarted 2024-04-25T16:50:10.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:09 smithi116 ceph-mon[32712]: daemon mds.k restarted 2024-04-25T16:50:10.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1228962 10.18 MB/sec execute 496 sec latency 21.809 ms 2024-04-25T16:50:11.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:10 smithi112 ceph-mon[29122]: pgmap v372: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-04-25T16:50:11.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:10 smithi112 ceph-mon[29122]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T16:50:11.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:10 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6832/868451243,v1:172.21.15.116:6833/868451243] up:boot 2024-04-25T16:50:11.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:10 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:50:11.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:10 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:50:11.095 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:10 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:50:11.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:10 smithi133 ceph-mon[32691]: pgmap v372: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-04-25T16:50:11.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:10 smithi133 ceph-mon[32691]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T16:50:11.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:10 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6832/868451243,v1:172.21.15.116:6833/868451243] up:boot 2024-04-25T16:50:11.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:10 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:50:11.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:10 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:50:11.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:10 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:50:11.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:10 smithi116 ceph-mon[32712]: pgmap v372: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-04-25T16:50:11.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:10 smithi116 ceph-mon[32712]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T16:50:11.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:10 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6832/868451243,v1:172.21.15.116:6833/868451243] up:boot 2024-04-25T16:50:11.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:10 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:50:11.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:10 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:50:11.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:10 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:50:11.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1230331 10.17 MB/sec execute 497 sec latency 23.649 ms 2024-04-25T16:50:12.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1232477 10.18 MB/sec execute 498 sec latency 17.480 ms 2024-04-25T16:50:13.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:13 smithi133 ceph-mon[32691]: pgmap v374: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-04-25T16:50:13.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:13 smithi133 ceph-mon[32691]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T16:50:13.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:13 smithi116 ceph-mon[32712]: pgmap v374: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-04-25T16:50:13.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:13 smithi116 ceph-mon[32712]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T16:50:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:13 smithi112 ceph-mon[29122]: pgmap v374: 97 pgs: 97 active+clean; 63 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-04-25T16:50:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:13 smithi112 ceph-mon[29122]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T16:50:13.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1233845 10.17 MB/sec execute 499 sec latency 17.373 ms 2024-04-25T16:50:14.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1235935 10.17 MB/sec execute 500 sec latency 61.966 ms 2024-04-25T16:50:15.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:15 smithi133 ceph-mon[32691]: pgmap v376: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2024-04-25T16:50:15.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:15 smithi116 ceph-mon[32712]: pgmap v376: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2024-04-25T16:50:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:15 smithi112 ceph-mon[29122]: pgmap v376: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 317 op/s 2024-04-25T16:50:15.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1236821 10.16 MB/sec execute 501 sec latency 62.528 ms 2024-04-25T16:50:16.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1238364 10.15 MB/sec execute 502 sec latency 219.514 ms 2024-04-25T16:50:17.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:17 smithi133 ceph-mon[32691]: pgmap v377: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2024-04-25T16:50:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:17 smithi116 ceph-mon[32712]: pgmap v377: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2024-04-25T16:50:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:17 smithi112 ceph-mon[29122]: pgmap v377: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 314 op/s 2024-04-25T16:50:17.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1239850 10.15 MB/sec execute 503 sec latency 17.155 ms 2024-04-25T16:50:18.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1241344 10.15 MB/sec execute 504 sec latency 34.049 ms 2024-04-25T16:50:19.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:19 smithi133 ceph-mon[32691]: pgmap v378: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-04-25T16:50:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:19 smithi116 ceph-mon[32712]: pgmap v378: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-04-25T16:50:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:19 smithi112 ceph-mon[29122]: pgmap v378: 97 pgs: 97 active+clean; 69 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-04-25T16:50:19.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1243171 10.15 MB/sec execute 505 sec latency 108.341 ms 2024-04-25T16:50:20.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1244587 10.14 MB/sec execute 506 sec latency 110.831 ms 2024-04-25T16:50:21.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:21 smithi133 ceph-mon[32691]: pgmap v379: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 -- 172.21.15.112:0/2751539308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58701042f0 msgr2=0x7f58701106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 --2- 172.21.15.112:0/2751539308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58701042f0 0x7f58701106c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5868009ca0 tx=0x7f586802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 -- 172.21.15.112:0/2751539308 shutdown_connections 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 --2- 172.21.15.112:0/2751539308 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58701042f0 0x7f58701106c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 --2- 172.21.15.112:0/2751539308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f587010a040 0x7f5870103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 --2- 172.21.15.112:0/2751539308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5870109ba0 0x7f5870103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 -- 172.21.15.112:0/2751539308 >> 172.21.15.112:0/2751539308 conn(0x7f58700792a0 msgr2=0x7f58700796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 -- 172.21.15.112:0/2751539308 shutdown_connections 2024-04-25T16:50:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.465+0000 7f5878586640 1 -- 172.21.15.112:0/2751539308 wait complete. 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 Processor -- start 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 -- start start 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 0x7f587019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5870109ba0 0x7f587019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f587010a040 0x7f587019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5870113db0 con 0x7f5870109ba0 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5870113c30 con 0x7f58701042f0 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5878586640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5870113f30 con 0x7f587010a040 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 0x7f587019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5875b21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5870109ba0 0x7f587019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 0x7f587019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:44616/0 (socket says 172.21.15.112:44616) 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 -- 172.21.15.112:0/186450558 learned_addr learned my addr 172.21.15.112:0/186450558 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876b23640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f587010a040 0x7f587019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 -- 172.21.15.112:0/186450558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f587010a040 msgr2=0x7f587019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f587010a040 0x7f587019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 -- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5870109ba0 msgr2=0x7f587019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5870109ba0 0x7f587019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.466+0000 7f5876322640 1 -- 172.21.15.112:0/186450558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587019f570 con 0x7f58701042f0 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.467+0000 7f5876322640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 0x7f587019a710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e9f0 tx=0x7f586c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.467+0000 7f58677fe640 1 -- 172.21.15.112:0/186450558 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c00ccc0 con 0x7f58701042f0 2024-04-25T16:50:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.467+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587019f860 con 0x7f58701042f0 2024-04-25T16:50:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.467+0000 7f58677fe640 1 -- 172.21.15.112:0/186450558 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586c0054d0 con 0x7f58701042f0 2024-04-25T16:50:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.467+0000 7f58677fe640 1 -- 172.21.15.112:0/186450558 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c010430 con 0x7f58701042f0 2024-04-25T16:50:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.467+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58701a71e0 con 0x7f58701042f0 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.469+0000 7f587f43f740 1 Processor -- start 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 -- start start 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f58677fe640 1 -- 172.21.15.112:0/186450558 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f586c0046c0 con 0x7f58701042f0 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95c6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595e952c460 con 0x5595e92261b0 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5595e94e6590 con 0x5595e92054d0 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f587f43f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595e9528420 con 0x5595e91fd2f0 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.475 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95c5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46162/0 (socket says 172.21.15.112:46162) 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5875b21640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 -- 172.21.15.112:0/1191051172 learned_addr learned my addr 172.21.15.112:0/1191051172 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876322640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 -- 172.21.15.112:0/1191051172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 msgr2=0x5595e95c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 -- 172.21.15.112:0/1191051172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 msgr2=0x5595e95c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5876b23640 1 -- 172.21.15.112:0/1191051172 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e94f30d0 con 0x5595e92261b0 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.470+0000 7f5875b21640 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f5876b23640 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95c5af0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f586802f410 tx=0x7f586802f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f8a11d0253bfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f58677fe640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f585c0afc50 0x7f585c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f5864bfe640 1 -- 172.21.15.112:0/1191051172 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868031cf0 con 0x5595e92261b0 2024-04-25T16:50:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f5876322640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f585c0afc50 0x7f585c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f5864bfe640 1 -- 172.21.15.112:0/1191051172 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5868036030 con 0x5595e92261b0 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f5864bfe640 1 -- 172.21.15.112:0/1191051172 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680042a0 con 0x5595e92261b0 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.471+0000 7f58677fe640 1 -- 172.21.15.112:0/186450558 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f586c0df410 con 0x7f58701042f0 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 -- 172.21.15.112:0/1191051172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 msgr2=0x5595e95c5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f5876322640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f585c0afc50 0x7f585c0b2160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f586c0dcbc0 tx=0x7f586c090040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95c5af0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f586802f410 tx=0x7f586802f440 comp rx=0 tx=0).stop 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 -- 172.21.15.112:0/1191051172 shutdown_connections 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 --2- 172.21.15.112:0/1191051172 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95c5af0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 -- 172.21.15.112:0/1191051172 >> 172.21.15.112:0/1191051172 conn(0x5595e921a300 msgr2=0x5595e95c22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 -- 172.21.15.112:0/1191051172 shutdown_connections 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 -- 172.21.15.112:0/1191051172 wait complete. 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 -- start start 2024-04-25T16:50:21.477 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.472+0000 7f587f43f740 1 Processor -- start 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e91fd2f0 0x5595e95d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92054d0 0x5595e95d06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5595e95d2a40 con 0x5595e92261b0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5595e94b3b60 con 0x5595e91fd2f0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5595e95d2bc0 con 0x5595e92054d0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f587f43f740 10 client.0 _ll_register_callbacks cb 0x5595e94d82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95d5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46168/0 (socket says 172.21.15.112:46168) 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 -- 172.21.15.112:0/4276407517 learned_addr learned my addr 172.21.15.112:0/4276407517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e91fd2f0 0x5595e95d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5875b21640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92054d0 0x5595e95d06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.478 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 -- 172.21.15.112:0/4276407517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92054d0 msgr2=0x5595e95d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92054d0 0x5595e95d06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 -- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e91fd2f0 msgr2=0x5595e95d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e91fd2f0 0x5595e95d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 -- 172.21.15.112:0/4276407517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e923e210 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5876b23640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95d5f30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5868009620 tx=0x7f5868003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4100b8a228b18e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.473+0000 7f5859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f5859ffb640 1 -- 172.21.15.112:0/4276407517 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868059060 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f5859ffb640 1 -- 172.21.15.112:0/4276407517 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f58680045e0 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5868036030 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f5859ffb640 1 -- 172.21.15.112:0/4276407517 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f58680047e0 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 10 client.16389 fetch_fsmap learned FSMap version 36 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5595e930a620 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f5859ffb640 1 -- 172.21.15.112:0/4276407517 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5868036030 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f5859ffb640 1 -- 172.21.15.112:0/4276407517 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f5868043070 con 0x5595e92261b0 2024-04-25T16:50:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 10 client.16389 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 20 client.16389 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91e4d80 0x5595e95e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 4 client.16389 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.474+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e953a070 con 0x5595e91e4d80 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.475+0000 7f5875b21640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91e4d80 0x5595e95e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.475+0000 7f5859ffb640 10 client.16389 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.475+0000 7f5875b21640 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91e4d80 0x5595e95e23d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e5a344ccee3a81 server_cookie=c744f5f3e3ce516c in_seq=0 out_seq=0 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f5859ffb640 1 -- 172.21.15.112:0/4276407517 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5595e953a070 con 0x5595e91e4d80 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f5859ffb640 10 client.16389 handle_command_reply: tid=0 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 client.16389 shutdown 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 msgr2=0x5595e95d5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95d5f30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5868009620 tx=0x7f5868003cc0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 shutdown_connections 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91e4d80 0x5595e95e23d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92054d0 0x5595e95d06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92261b0 0x5595e95d5f30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f5859ffb640 0 client.16389 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 --2- 172.21.15.112:0/4276407517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e91fd2f0 0x5595e95d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 >> 172.21.15.112:0/4276407517 conn(0x5595e921a300 msgr2=0x5595e95c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 shutdown_connections 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 1 -- 172.21.15.112:0/4276407517 wait complete. 2024-04-25T16:50:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.476+0000 7f587f43f740 20 client.16389 trim_cache size 0 max 16384 2024-04-25T16:50:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.491+0000 7f587f43f740 1 Processor -- start 2024-04-25T16:50:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.491+0000 7f587f43f740 1 -- start start 2024-04-25T16:50:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.491+0000 7f587f43f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 0x5595e95c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.491+0000 7f587f43f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.491+0000 7f587f43f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92261b0 0x5595e95cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f587f43f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58701b23d0 con 0x5595e91fd2f0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f587f43f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f58701afca0 con 0x5595e92054d0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f587f43f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58701adb10 con 0x5595e92261b0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 0x5595e95c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 0x5595e95c5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46170/0 (socket says 172.21.15.112:46170) 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 -- 172.21.15.112:0/3072640602 learned_addr learned my addr 172.21.15.112:0/3072640602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5875b21640 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92261b0 0x5595e95cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 -- 172.21.15.112:0/3072640602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92261b0 msgr2=0x5595e95cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92261b0 0x5595e95cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 -- 172.21.15.112:0/3072640602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 msgr2=0x5595e95d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 -- 172.21.15.112:0/3072640602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e95ca1e0 con 0x5595e91fd2f0 2024-04-25T16:50:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5875b21640 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f5876b23640 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 0x5595e95c5f90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f586802f6c0 tx=0x7f586802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ec1011fa794305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f585bfff640 1 -- 172.21.15.112:0/3072640602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5868059020 con 0x5595e91fd2f0 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f585bfff640 1 -- 172.21.15.112:0/3072640602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5868036070 con 0x5595e91fd2f0 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.492+0000 7f585bfff640 1 -- 172.21.15.112:0/3072640602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586802fce0 con 0x5595e91fd2f0 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 -- 172.21.15.112:0/3072640602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 msgr2=0x5595e95c5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 0x5595e95c5f90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f586802f6c0 tx=0x7f586802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 -- 172.21.15.112:0/3072640602 shutdown_connections 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e92261b0 0x5595e95cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92054d0 0x5595e95d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 --2- 172.21.15.112:0/3072640602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e91fd2f0 0x5595e95c5f90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 -- 172.21.15.112:0/3072640602 >> 172.21.15.112:0/3072640602 conn(0x5595e921a300 msgr2=0x5595e95c5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 -- 172.21.15.112:0/3072640602 shutdown_connections 2024-04-25T16:50:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.493+0000 7f587f43f740 1 -- 172.21.15.112:0/3072640602 wait complete. 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 -- start start 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 Processor -- start 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 0x5595e95dbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92261b0 0x5595e9642540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f58701ae160 con 0x5595e92054d0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f58701af5d0 con 0x5595e92261b0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58701ae830 con 0x5595e91fd2f0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f587f43f740 10 client.0 _ll_register_callbacks cb 0x7f5860009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 0x5595e95dbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 0x5595e95dbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46182/0 (socket says 172.21.15.112:46182) 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 -- 172.21.15.112:0/3858888307 learned_addr learned my addr 172.21.15.112:0/3858888307 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5876b23640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5876b23640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92261b0 0x5595e9642540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 -- 172.21.15.112:0/3858888307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 msgr2=0x5595e95db880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95db880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 -- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92261b0 msgr2=0x5595e9642540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92261b0 0x5595e9642540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5875b21640 1 -- 172.21.15.112:0/3858888307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e9642a80 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.494+0000 7f5876b23640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5875b21640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 0x5595e95dbdc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f587010a470 tx=0x7f58600207e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4397c096316bb94a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5859ffb640 1 -- 172.21.15.112:0/3858888307 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860003d20 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5859ffb640 1 -- 172.21.15.112:0/3858888307 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f5860010430 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5868036070 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5859ffb640 1 -- 172.21.15.112:0/3858888307 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5860010630 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 10 client.16395 fetch_fsmap learned FSMap version 36 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5595e9649de0 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5859ffb640 1 -- 172.21.15.112:0/3858888307 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5868036070 con 0x5595e92054d0 2024-04-25T16:50:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f5859ffb640 1 -- 172.21.15.112:0/3858888307 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f58600287d0 con 0x5595e92054d0 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 10 client.16395 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 10 client.16395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 20 client.16395 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91ba1c0 0x5595e964c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 4 client.16395 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.495+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595e9533ee0 con 0x5595e91ba1c0 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.496+0000 7f5876b23640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91ba1c0 0x5595e964c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.496+0000 7f5859ffb640 10 client.16395 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.496+0000 7f5876b23640 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91ba1c0 0x5595e964c550 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3058bcf962963e5 server_cookie=a48763a8d72689c in_seq=0 out_seq=0 2024-04-25T16:50:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:21 smithi116 ceph-mon[32712]: pgmap v379: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-04-25T16:50:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:21 smithi112 ceph-mon[29122]: pgmap v379: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 192 op/s 2024-04-25T16:50:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.663+0000 7f5859ffb640 1 -- 172.21.15.112:0/3858888307 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595e9533ee0 con 0x5595e91ba1c0 2024-04-25T16:50:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.663+0000 7f5859ffb640 10 client.16395 handle_command_reply: tid=0 2024-04-25T16:50:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 client.16395 shutdown 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 msgr2=0x5595e95dbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 0x5595e95dbdc0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f587010a470 tx=0x7f58600207e0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 shutdown_connections 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595e92261b0 0x5595e9642540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5595e92054d0 0x5595e95dbdc0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5595e91ba1c0 0x5595e964c550 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f5859ffb640 0 client.16395 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 --2- 172.21.15.112:0/3858888307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5595e91fd2f0 0x5595e95db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 >> 172.21.15.112:0/3858888307 conn(0x5595e921a300 msgr2=0x5595e95cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 shutdown_connections 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 1 -- 172.21.15.112:0/3858888307 wait complete. 2024-04-25T16:50:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.664+0000 7f587f43f740 20 client.16395 trim_cache size 0 max 16384 2024-04-25T16:50:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:50:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:50:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:50:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f585c0afc50 msgr2=0x7f585c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f585c0afc50 0x7f585c0b2160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f586c0dcbc0 tx=0x7f586c090040 comp rx=0 tx=0).stop 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 msgr2=0x7f587019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 0x7f587019a710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f586c00e9f0 tx=0x7f586c00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 shutdown_connections 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f585c0afc50 0x7f585c0b2160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f587010a040 0x7f587019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5870109ba0 0x7f587019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 --2- 172.21.15.112:0/186450558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58701042f0 0x7f587019a710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.665+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 >> 172.21.15.112:0/186450558 conn(0x7f58700792a0 msgr2=0x7f5870100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.666+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 shutdown_connections 2024-04-25T16:50:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:21.666+0000 7f5878586640 1 -- 172.21.15.112:0/186450558 wait complete. 2024-04-25T16:50:21.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1246578 10.15 MB/sec execute 507 sec latency 121.524 ms 2024-04-25T16:50:22.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1247552 10.13 MB/sec execute 508 sec latency 35.573 ms 2024-04-25T16:50:23.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:23 smithi133 ceph-mon[32691]: pgmap v380: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-04-25T16:50:23.480 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:23 smithi112 ceph-mon[29122]: pgmap v380: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-04-25T16:50:23.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:23 smithi116 ceph-mon[32712]: pgmap v380: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 191 op/s 2024-04-25T16:50:23.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1249942 10.14 MB/sec execute 509 sec latency 25.930 ms 2024-04-25T16:50:24.895 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1251019 10.13 MB/sec execute 510 sec latency 15.840 ms 2024-04-25T16:50:25.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:25 smithi133 ceph-mon[32691]: pgmap v381: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 324 op/s 2024-04-25T16:50:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:25 smithi116 ceph-mon[32712]: pgmap v381: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 324 op/s 2024-04-25T16:50:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:25 smithi112 ceph-mon[29122]: pgmap v381: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.1 MiB/s wr, 324 op/s 2024-04-25T16:50:25.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1252893 10.13 MB/sec execute 511 sec latency 50.534 ms 2024-04-25T16:50:26.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1254284 10.13 MB/sec execute 512 sec latency 13.275 ms 2024-04-25T16:50:27.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:27 smithi133 ceph-mon[32691]: pgmap v382: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-04-25T16:50:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:27 smithi116 ceph-mon[32712]: pgmap v382: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-04-25T16:50:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:27 smithi112 ceph-mon[29122]: pgmap v382: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-04-25T16:50:27.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1256281 10.12 MB/sec execute 513 sec latency 31.443 ms 2024-04-25T16:50:28.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1257742 10.12 MB/sec execute 514 sec latency 44.665 ms 2024-04-25T16:50:29.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:29 smithi133 ceph-mon[32691]: pgmap v383: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-04-25T16:50:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:29 smithi116 ceph-mon[32712]: pgmap v383: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-04-25T16:50:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:29 smithi112 ceph-mon[29122]: pgmap v383: 97 pgs: 97 active+clean; 71 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 233 op/s 2024-04-25T16:50:29.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1259765 10.12 MB/sec execute 515 sec latency 28.167 ms 2024-04-25T16:50:30.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1261064 10.12 MB/sec execute 516 sec latency 39.345 ms 2024-04-25T16:50:31.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:31 smithi133 ceph-mon[32691]: pgmap v384: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-04-25T16:50:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:31 smithi116 ceph-mon[32712]: pgmap v384: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-04-25T16:50:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:31 smithi112 ceph-mon[29122]: pgmap v384: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-04-25T16:50:31.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1262722 10.11 MB/sec execute 517 sec latency 21.926 ms 2024-04-25T16:50:32.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1264538 10.12 MB/sec execute 518 sec latency 17.653 ms 2024-04-25T16:50:33.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:33 smithi133 ceph-mon[32691]: pgmap v385: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 230 op/s 2024-04-25T16:50:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:33 smithi116 ceph-mon[32712]: pgmap v385: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 230 op/s 2024-04-25T16:50:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:33 smithi112 ceph-mon[29122]: pgmap v385: 97 pgs: 97 active+clean; 77 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 230 op/s 2024-04-25T16:50:33.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1265990 10.11 MB/sec execute 519 sec latency 26.910 ms 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 -- 172.21.15.112:0/1866274823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 msgr2=0x7f46a8102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 --2- 172.21.15.112:0/1866274823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 0x7f46a8102b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4694009900 tx=0x7f469402a410 comp rx=0 tx=0).stop 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 -- 172.21.15.112:0/1866274823 shutdown_connections 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 --2- 172.21.15.112:0/1866274823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a810f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 --2- 172.21.15.112:0/1866274823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 0x7f46a8102b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 --2- 172.21.15.112:0/1866274823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46a8108a60 0x7f46a8102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 -- 172.21.15.112:0/1866274823 >> 172.21.15.112:0/1866274823 conn(0x7f46a80781b0 msgr2=0x7f46a80785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 -- 172.21.15.112:0/1866274823 shutdown_connections 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.768+0000 7f46af1a8640 1 -- 172.21.15.112:0/1866274823 wait complete. 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 Processor -- start 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 -- start start 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46a8108a60 0x7f46a819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 0x7f46a819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112c80 con 0x7f46a8108f00 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112b00 con 0x7f46a81031c0 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46af1a8640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46a8112e00 con 0x7f46a8108a60 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:47350/0 (socket says 172.21.15.112:47350) 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 -- 172.21.15.112:0/3014555254 learned_addr learned my addr 172.21.15.112:0/3014555254 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46ad745640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 0x7f46a819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46a7fff640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46a8108a60 0x7f46a819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 -- 172.21.15.112:0/3014555254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46a8108a60 msgr2=0x7f46a819ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46a8108a60 0x7f46a819ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 -- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 msgr2=0x7f46a819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 0x7f46a819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.769+0000 7f46acf44640 1 -- 172.21.15.112:0/3014555254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a819f610 con 0x7f46a81031c0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.770+0000 7f46acf44640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a819a7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e9c0 tx=0x7f469c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.770+0000 7f46a5ffb640 1 -- 172.21.15.112:0/3014555254 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c00ccc0 con 0x7f46a81031c0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.770+0000 7f46a5ffb640 1 -- 172.21.15.112:0/3014555254 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469c0054d0 con 0x7f46a81031c0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.770+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a819f900 con 0x7f46a81031c0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.770+0000 7f46a5ffb640 1 -- 172.21.15.112:0/3014555254 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c010430 con 0x7f46a81031c0 2024-04-25T16:50:34.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.770+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a819fdd0 con 0x7f46a81031c0 2024-04-25T16:50:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.773+0000 7f46b605b740 1 Processor -- start 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.773+0000 7f46a5ffb640 1 -- 172.21.15.112:0/3014555254 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f469c005020 con 0x7f46a81031c0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.773+0000 7f46b605b740 1 -- start start 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46b605b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 0x55959d1c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46b605b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46b605b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46b605b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55959d12d460 con 0x55959ce271b0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46b605b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55959d0e7590 con 0x55959ce064d0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46b605b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55959d129420 con 0x55959cdfe2f0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 0x55959d1c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46acf44640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 0x55959d1c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47320/0 (socket says 172.21.15.112:47320) 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 -- 172.21.15.112:0/3466792121 learned_addr learned my addr 172.21.15.112:0/3466792121 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46a7fff640 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 -- 172.21.15.112:0/3466792121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 msgr2=0x55959d1c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46a5ffb640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46980afc50 0x7f46980b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 -- 172.21.15.112:0/3466792121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 msgr2=0x55959d1c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 -- 172.21.15.112:0/3466792121 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959d0f40d0 con 0x55959ce271b0 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46acf44640 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46a5ffb640 1 -- 172.21.15.112:0/3014555254 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f469c0df390 con 0x7f46a81031c0 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46980afc50 0x7f46980b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f46ad745640 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 0x55959d1c6af0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009fd0 tx=0x7f46a00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe86a98e7f073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.774+0000 7f4692ffd640 1 -- 172.21.15.112:0/3466792121 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0004250 con 0x55959ce271b0 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f4692ffd640 1 -- 172.21.15.112:0/3466792121 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a0003db0 con 0x55959ce271b0 2024-04-25T16:50:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f4692ffd640 1 -- 172.21.15.112:0/3466792121 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a001bb80 con 0x55959ce271b0 2024-04-25T16:50:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 -- 172.21.15.112:0/3466792121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 msgr2=0x55959d1c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 0x55959d1c6af0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009fd0 tx=0x7f46a00118d0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46ad745640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46980afc50 0x7f46980b2160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f46a0020000 tx=0x7f46a0021000 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 -- 172.21.15.112:0/3466792121 shutdown_connections 2024-04-25T16:50:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 --2- 172.21.15.112:0/3466792121 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce271b0 0x55959d1c6af0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.775+0000 7f46b605b740 1 -- 172.21.15.112:0/3466792121 >> 172.21.15.112:0/3466792121 conn(0x55959ce1b300 msgr2=0x55959d1c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 -- 172.21.15.112:0/3466792121 shutdown_connections 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 -- 172.21.15.112:0/3466792121 wait complete. 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 -- start start 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 Processor -- start 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d1d1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959ce064d0 0x55959d1d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce271b0 0x55959d1db520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55959d1d3c20 con 0x55959cdfe2f0 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55959d0b4b60 con 0x55959ce271b0 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55959d1d3da0 con 0x55959ce064d0 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46b605b740 10 client.0 _ll_register_callbacks cb 0x55959d0d92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.776+0000 7f46acf44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d1d1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d1d1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47338/0 (socket says 172.21.15.112:47338) 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 -- 172.21.15.112:0/554209743 learned_addr learned my addr 172.21.15.112:0/554209743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46ad745640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce271b0 0x55959d1db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46a7fff640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959ce064d0 0x55959d1d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 -- 172.21.15.112:0/554209743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959ce064d0 msgr2=0x55959d1d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959ce064d0 0x55959d1d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 -- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce271b0 msgr2=0x55959d1db520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce271b0 0x55959d1db520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 -- 172.21.15.112:0/554209743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959ce3f210 con 0x55959cdfe2f0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46acf44640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d1d1170 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f469c004650 tx=0x7f469c0a33a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76719de2e0a48e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f4687fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f4687fff640 1 -- 172.21.15.112:0/554209743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c0a6c50 con 0x55959cdfe2f0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f4687fff640 1 -- 172.21.15.112:0/554209743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f469c0afeb0 con 0x55959cdfe2f0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a0003db0 con 0x55959cdfe2f0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f4687fff640 1 -- 172.21.15.112:0/554209743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f469c00ce60 con 0x55959cdfe2f0 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46b605b740 10 client.16407 fetch_fsmap learned FSMap version 36 2024-04-25T16:50:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55959cf0b620 con 0x55959cdfe2f0 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.777+0000 7f4687fff640 1 -- 172.21.15.112:0/554209743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a0003db0 con 0x55959cdfe2f0 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f4687fff640 1 -- 172.21.15.112:0/554209743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f469c0a2e50 con 0x55959cdfe2f0 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46b605b740 10 client.16407 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46b605b740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46b605b740 20 client.16407 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46b605b740 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959cde5d80 0x55959d1e7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46b605b740 4 client.16407 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55959d13b070 con 0x55959cde5d80 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46a7fff640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959cde5d80 0x55959d1e7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f4687fff640 10 client.16407 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.778+0000 7f46a7fff640 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959cde5d80 0x55959d1e7b50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b58a549243575fc server_cookie=92cdf62654813cc2 in_seq=0 out_seq=0 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f4687fff640 1 -- 172.21.15.112:0/554209743 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55959d13b070 con 0x55959cde5d80 2024-04-25T16:50:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f4687fff640 10 client.16407 handle_command_reply: tid=0 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 client.16407 shutdown 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 msgr2=0x55959d1d1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d1d1170 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f469c004650 tx=0x7f469c0a33a0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 shutdown_connections 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959cde5d80 0x55959d1e7b50 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959ce064d0 0x55959d1d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce271b0 0x55959d1db520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f4687fff640 0 client.16407 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 --2- 172.21.15.112:0/554209743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d1d1170 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.780+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 >> 172.21.15.112:0/554209743 conn(0x55959ce1b300 msgr2=0x55959d1c6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.781+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 shutdown_connections 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.781+0000 7f46b605b740 1 -- 172.21.15.112:0/554209743 wait complete. 2024-04-25T16:50:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.781+0000 7f46b605b740 20 client.16407 trim_cache size 0 max 16384 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 Processor -- start 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 -- start start 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959cde5d80 0x55959d1db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 0x55959d1ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81b01c0 con 0x55959ce064d0 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81ada90 con 0x55959cde5d80 2024-04-25T16:50:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46b605b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81ab900 con 0x55959cdfe2f0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46ad745640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 0x55959d1ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46ad745640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 0x55959d1ed330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47346/0 (socket says 172.21.15.112:47346) 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46ad745640 1 -- 172.21.15.112:0/3563714367 learned_addr learned my addr 172.21.15.112:0/3563714367 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46acf44640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959cde5d80 0x55959d1db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.797+0000 7f46a7fff640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46ad745640 1 -- 172.21.15.112:0/3563714367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 msgr2=0x55959d1c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46ad745640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46ad745640 1 -- 172.21.15.112:0/3563714367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959cde5d80 msgr2=0x55959d1db4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46ad745640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959cde5d80 0x55959d1db4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46ad745640 1 -- 172.21.15.112:0/3563714367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55959d1cf4b0 con 0x55959ce064d0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46acf44640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959cde5d80 0x55959d1db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46a7fff640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46ad745640 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 0x55959d1ed330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55959d1c3c60 tx=0x7f46a0027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0d5c27dc06d3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46927fc640 1 -- 172.21.15.112:0/3563714367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0036070 con 0x55959ce064d0 2024-04-25T16:50:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46927fc640 1 -- 172.21.15.112:0/3563714367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a000fe30 con 0x55959ce064d0 2024-04-25T16:50:34.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.798+0000 7f46927fc640 1 -- 172.21.15.112:0/3563714367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0029040 con 0x55959ce064d0 2024-04-25T16:50:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 -- 172.21.15.112:0/3563714367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 msgr2=0x55959d1ed330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 0x55959d1ed330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55959d1c3c60 tx=0x7f46a0027000 comp rx=0 tx=0).stop 2024-04-25T16:50:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 -- 172.21.15.112:0/3563714367 shutdown_connections 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cdfe2f0 0x55959d1c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959ce064d0 0x55959d1ed330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 --2- 172.21.15.112:0/3563714367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959cde5d80 0x55959d1db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 -- 172.21.15.112:0/3563714367 >> 172.21.15.112:0/3563714367 conn(0x55959ce1b300 msgr2=0x55959d1c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 -- 172.21.15.112:0/3563714367 shutdown_connections 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 -- 172.21.15.112:0/3563714367 wait complete. 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 -- start start 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.799+0000 7f46b605b740 1 Processor -- start 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cde5d80 0x55959d248310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d248850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81abf50 con 0x55959cdfe2f0 2024-04-25T16:50:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81ad3c0 con 0x55959ce064d0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46a81ac620 con 0x55959cde5d80 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46b605b740 10 client.0 _ll_register_callbacks cb 0x7f469c0e2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d248850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d248850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47348/0 (socket says 172.21.15.112:47348) 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46ad745640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 -- 172.21.15.112:0/1953414223 learned_addr learned my addr 172.21.15.112:0/1953414223 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46acf44640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cde5d80 0x55959d248310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 -- 172.21.15.112:0/1953414223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cde5d80 msgr2=0x55959d248310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cde5d80 0x55959d248310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 -- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 msgr2=0x55959d1dea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1dea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 -- 172.21.15.112:0/1953414223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55959d1df030 con 0x55959cdfe2f0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46ad745640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f46a7fff640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d248850 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4694005940 tx=0x7f46940042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ebbcc6ffc0bf8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:34.803 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.800+0000 7f4687fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f4687fff640 1 -- 172.21.15.112:0/1953414223 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694035040 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f4687fff640 1 -- 172.21.15.112:0/1953414223 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f4694005240 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a000fe30 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f4687fff640 1 -- 172.21.15.112:0/1953414223 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4694055050 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 10 client.16413 fetch_fsmap learned FSMap version 36 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55959d24f9c0 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f4687fff640 1 -- 172.21.15.112:0/1953414223 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46a000fe30 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f4687fff640 1 -- 172.21.15.112:0/1953414223 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f46940463b0 con 0x55959cdfe2f0 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 10 client.16413 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 20 client.16413 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959ce271b0 0x55959d252200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 4 client.16413 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:50:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55959d134ee0 con 0x55959ce271b0 2024-04-25T16:50:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.801+0000 7f46acf44640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959ce271b0 0x55959d252200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.802+0000 7f4687fff640 10 client.16413 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:34.802+0000 7f46acf44640 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959ce271b0 0x55959d252200 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e607530dbda1a87d server_cookie=78ed02ea3d029b92 in_seq=0 out_seq=0 2024-04-25T16:50:34.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1267939 10.11 MB/sec execute 520 sec latency 63.427 ms 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f4687fff640 1 -- 172.21.15.112:0/1953414223 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55959d134ee0 con 0x55959ce271b0 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f4687fff640 10 client.16413 handle_command_reply: tid=0 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 client.16413 shutdown 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 msgr2=0x55959d248850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d248850 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4694005940 tx=0x7f46940042c0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 shutdown_connections 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55959ce064d0 0x55959d1dea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55959cdfe2f0 0x55959d248850 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55959ce271b0 0x55959d252200 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 --2- 172.21.15.112:0/1953414223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55959cde5d80 0x55959d248310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 >> 172.21.15.112:0/1953414223 conn(0x55959ce1b300 msgr2=0x55959d1d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f4687fff640 0 client.16413 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 shutdown_connections 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 1 -- 172.21.15.112:0/1953414223 wait complete. 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.019+0000 7f46b605b740 20 client.16413 trim_cache size 0 max 16384 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:50:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:50:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46980afc50 msgr2=0x7f46980b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46980afc50 0x7f46980b2160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f46a0020000 tx=0x7f46a0021000 comp rx=0 tx=0).stop 2024-04-25T16:50:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 msgr2=0x7f46a819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a819a7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f469c00e9c0 tx=0x7f469c00ee80 comp rx=0 tx=0).stop 2024-04-25T16:50:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 shutdown_connections 2024-04-25T16:50:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f46980afc50 0x7f46980b2160 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46a8108a60 0x7f46a819ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46a8108f00 0x7f46a819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 --2- 172.21.15.112:0/3014555254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46a81031c0 0x7f46a819a7b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 >> 172.21.15.112:0/3014555254 conn(0x7f46a80781b0 msgr2=0x7f46a806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 shutdown_connections 2024-04-25T16:50:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:35.020+0000 7f46af1a8640 1 -- 172.21.15.112:0/3014555254 wait complete. 2024-04-25T16:50:35.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:35 smithi133 ceph-mon[32691]: pgmap v386: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-04-25T16:50:35.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:35 smithi116 ceph-mon[32712]: pgmap v386: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-04-25T16:50:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:35 smithi112 ceph-mon[29122]: pgmap v386: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.8 MiB/s wr, 312 op/s 2024-04-25T16:50:35.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1268824 10.10 MB/sec execute 521 sec latency 68.338 ms 2024-04-25T16:50:36.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1271387 10.11 MB/sec execute 522 sec latency 15.349 ms 2024-04-25T16:50:37.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:37 smithi133 ceph-mon[32691]: pgmap v387: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-25T16:50:37.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:37 smithi116 ceph-mon[32712]: pgmap v387: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-25T16:50:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:37 smithi112 ceph-mon[29122]: pgmap v387: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-25T16:50:37.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1272552 10.10 MB/sec execute 523 sec latency 17.144 ms 2024-04-25T16:50:38.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1274933 10.11 MB/sec execute 524 sec latency 18.956 ms 2024-04-25T16:50:39.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:39 smithi133 ceph-mon[32691]: pgmap v388: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-25T16:50:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:39 smithi116 ceph-mon[32712]: pgmap v388: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-25T16:50:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:39 smithi112 ceph-mon[29122]: pgmap v388: 97 pgs: 97 active+clean; 83 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 158 op/s 2024-04-25T16:50:39.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1275852 10.09 MB/sec execute 525 sec latency 33.745 ms 2024-04-25T16:50:40.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1277946 10.10 MB/sec execute 526 sec latency 95.006 ms 2024-04-25T16:50:41.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:41 smithi133 ceph-mon[32691]: pgmap v389: 97 pgs: 97 active+clean; 73 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 309 op/s 2024-04-25T16:50:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:41 smithi116 ceph-mon[32712]: pgmap v389: 97 pgs: 97 active+clean; 73 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 309 op/s 2024-04-25T16:50:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:41 smithi112 ceph-mon[29122]: pgmap v389: 97 pgs: 97 active+clean; 73 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 309 op/s 2024-04-25T16:50:41.896 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1279147 10.09 MB/sec execute 527 sec latency 22.162 ms 2024-04-25T16:50:42.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1281477 10.10 MB/sec execute 528 sec latency 14.623 ms 2024-04-25T16:50:43.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:43 smithi133 ceph-mon[32691]: pgmap v390: 97 pgs: 97 active+clean; 73 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-04-25T16:50:43.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:43 smithi116 ceph-mon[32712]: pgmap v390: 97 pgs: 97 active+clean; 73 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-04-25T16:50:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:43 smithi112 ceph-mon[29122]: pgmap v390: 97 pgs: 97 active+clean; 73 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-04-25T16:50:43.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1282695 10.09 MB/sec execute 529 sec latency 31.186 ms 2024-04-25T16:50:44.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1284927 10.09 MB/sec execute 530 sec latency 17.498 ms 2024-04-25T16:50:45.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:45 smithi133 ceph-mon[32691]: pgmap v391: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 308 op/s 2024-04-25T16:50:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:45 smithi116 ceph-mon[32712]: pgmap v391: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 308 op/s 2024-04-25T16:50:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:45 smithi112 ceph-mon[29122]: pgmap v391: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.3 MiB/s wr, 308 op/s 2024-04-25T16:50:45.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1286050 10.09 MB/sec execute 531 sec latency 112.680 ms 2024-04-25T16:50:46.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1287962 10.08 MB/sec execute 532 sec latency 32.494 ms 2024-04-25T16:50:47.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:47 smithi133 ceph-mon[32691]: pgmap v392: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-04-25T16:50:47.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:47 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:50:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:47 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:47 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:47 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:47 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:47 smithi116 ceph-mon[32712]: pgmap v392: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-04-25T16:50:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:47 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:50:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:47 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:47 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:47 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:47 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:47 smithi112 ceph-mon[29122]: pgmap v392: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-04-25T16:50:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:47 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:50:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:47 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:47 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:47 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:47 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:47.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1289428 10.08 MB/sec execute 533 sec latency 55.485 ms 2024-04-25T16:50:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1026508763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 msgr2=0x7f7e9410f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1026508763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9410f5b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009ca0 tx=0x7f7e8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1026508763 shutdown_connections 2024-04-25T16:50:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1026508763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e94108f30 0x7f7e94102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1026508763 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9410f5b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1026508763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108a90 0x7f7e94102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1026508763 >> 172.21.15.112:0/1026508763 conn(0x7f7e940fe260 msgr2=0x7f7e94100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1026508763 shutdown_connections 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.120+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1026508763 wait complete. 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 Processor -- start 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 -- start start 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e94108a90 0x7f7e9419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108f30 0x7f7e9419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94112ca0 con 0x7f7e941031e0 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94112b20 con 0x7f7e94108f30 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e9ae96640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94112e20 con 0x7f7e94108a90 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37612/0 (socket says 172.21.15.112:37612) 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 -- 172.21.15.112:0/1674545087 learned_addr learned my addr 172.21.15.112:0/1674545087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e99433640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108f30 0x7f7e9419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e93fff640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e94108a90 0x7f7e9419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 -- 172.21.15.112:0/1674545087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e94108a90 msgr2=0x7f7e9419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e94108a90 0x7f7e9419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 -- 172.21.15.112:0/1674545087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108f30 msgr2=0x7f7e9419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108f30 0x7f7e9419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 -- 172.21.15.112:0/1674545087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9419f570 con 0x7f7e941031e0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e99433640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108f30 0x7f7e9419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.121+0000 7f7e98c32640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9419a710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e9c0 tx=0x7f7e8800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.122+0000 7f7e91ffb640 1 -- 172.21.15.112:0/1674545087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800cd80 con 0x7f7e941031e0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.122+0000 7f7e91ffb640 1 -- 172.21.15.112:0/1674545087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e880054d0 con 0x7f7e941031e0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.122+0000 7f7e91ffb640 1 -- 172.21.15.112:0/1674545087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88010430 con 0x7f7e941031e0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.122+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9419f860 con 0x7f7e941031e0 2024-04-25T16:50:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.122+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9419fd60 con 0x7f7e941031e0 2024-04-25T16:50:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.124+0000 7f7e91ffb640 1 -- 172.21.15.112:0/1674545087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f7e88004fd0 con 0x7f7e941031e0 2024-04-25T16:50:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 Processor -- start 2024-04-25T16:50:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 -- start start 2024-04-25T16:50:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e91ffb640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e840afcd0 0x7f7e840b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x5561837feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 0x5561837ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561837ff570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e99433640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e840afcd0 0x7f7e840b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556183765460 con 0x5561834362f0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55618371f590 con 0x55618343e4d0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556183761420 con 0x55618345f1b0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e91ffb640 1 -- 172.21.15.112:0/1674545087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f7e880df430 con 0x7f7e941031e0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 0x5561837ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 0x5561837ff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37626/0 (socket says 172.21.15.112:37626) 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 -- 172.21.15.112:0/3023458933 learned_addr learned my addr 172.21.15.112:0/3023458933 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e99433640 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561837ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e98c32640 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x5561837feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 -- 172.21.15.112:0/3023458933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 msgr2=0x5561837feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x5561837feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 -- 172.21.15.112:0/3023458933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 msgr2=0x5561837ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561837ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e93fff640 1 -- 172.21.15.112:0/3023458933 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55618372c0d0 con 0x5561834362f0 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.125+0000 7f7e99433640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e840afcd0 0x7f7e840b21e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c02f500 tx=0x7f7e8c009980 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.126+0000 7f7e93fff640 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 0x5561837ff030 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009c00 tx=0x7f7e80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1e3bfd2571833a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.126+0000 7f7e7effd640 1 -- 172.21.15.112:0/3023458933 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80004030 con 0x5561834362f0 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.126+0000 7f7e7effd640 1 -- 172.21.15.112:0/3023458933 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e800041d0 con 0x5561834362f0 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.126+0000 7f7e7effd640 1 -- 172.21.15.112:0/3023458933 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80019730 con 0x5561834362f0 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3023458933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 msgr2=0x5561837ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 0x5561837ff030 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009c00 tx=0x7f7e80011810 comp rx=0 tx=0).stop 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3023458933 shutdown_connections 2024-04-25T16:50:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561837ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561834362f0 0x5561837ff030 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3023458933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x5561837feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3023458933 >> 172.21.15.112:0/3023458933 conn(0x556183453300 msgr2=0x5561837fb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3023458933 shutdown_connections 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3023458933 wait complete. 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 -- start start 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.127+0000 7f7ea1e28740 1 Processor -- start 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561834362f0 0x5561838091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561838096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 0x55618380f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55618380bc20 con 0x55618345f1b0 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5561836ecb60 con 0x55618343e4d0 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55618380bda0 con 0x5561834362f0 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 10 client.0 _ll_register_callbacks cb 0x5561837112b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 0x55618380f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561838096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 0x55618380f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37628/0 (socket says 172.21.15.112:37628) 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 -- 172.21.15.112:0/3079887061 learned_addr learned my addr 172.21.15.112:0/3079887061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e98c32640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561834362f0 0x5561838091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 -- 172.21.15.112:0/3079887061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561834362f0 msgr2=0x5561838091b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561834362f0 0x5561838091b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 -- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 msgr2=0x5561838096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561838096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 -- 172.21.15.112:0/3079887061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556183477210 con 0x55618345f1b0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e93fff640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561838096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e99433640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 0x55618380f110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c035000 tx=0x7f7e8c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a5662df11b86b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e73fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e73fff640 1 -- 172.21.15.112:0/3079887061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c004300 con 0x55618345f1b0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e73fff640 1 -- 172.21.15.112:0/3079887061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f7e8c0044a0 con 0x55618345f1b0 2024-04-25T16:50:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e800041d0 con 0x55618345f1b0 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7e73fff640 1 -- 172.21.15.112:0/3079887061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c0046a0 con 0x55618345f1b0 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 10 client.16425 fetch_fsmap learned FSMap version 36 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.128+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556183543620 con 0x55618345f1b0 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7e73fff640 1 -- 172.21.15.112:0/3079887061 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e800041d0 con 0x55618345f1b0 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7e73fff640 1 -- 172.21.15.112:0/3079887061 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f7e8c02fd50 con 0x55618345f1b0 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7ea1e28740 10 client.16425 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7ea1e28740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7ea1e28740 20 client.16425 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55618341dd80 0x55618381b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7ea1e28740 4 client.16425 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556183773070 con 0x55618341dd80 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7e98c32640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55618341dd80 0x55618381b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7e73fff640 10 client.16425 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:48.133 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.129+0000 7f7e98c32640 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55618341dd80 0x55618381b760 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fab36f764a044d5 server_cookie=91a8ff38957610c9 in_seq=0 out_seq=0 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.131+0000 7f7e73fff640 1 -- 172.21.15.112:0/3079887061 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556183773070 con 0x55618341dd80 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.131+0000 7f7e73fff640 10 client.16425 handle_command_reply: tid=0 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 client.16425 shutdown 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 msgr2=0x55618380f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 0x55618380f110 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c035000 tx=0x7f7e8c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 shutdown_connections 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618345f1b0 0x55618380f110 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55618343e4d0 0x5561838096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55618341dd80 0x55618381b760 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/3079887061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5561834362f0 0x5561838091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7e73fff640 0 client.16425 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 >> 172.21.15.112:0/3079887061 conn(0x556183453300 msgr2=0x5561837fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 shutdown_connections 2024-04-25T16:50:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 1 -- 172.21.15.112:0/3079887061 wait complete. 2024-04-25T16:50:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.132+0000 7f7ea1e28740 20 client.16425 trim_cache size 0 max 16384 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 Processor -- start 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 -- start start 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183802ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 0x556183820f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183821460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941b23f0 con 0x55618343e4d0 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941afcc0 con 0x5561834362f0 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941adb30 con 0x55618345f1b0 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 0x556183820f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7e93fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 0x556183820f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37642/0 (socket says 172.21.15.112:37642) 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7e93fff640 1 -- 172.21.15.112:0/2794057574 learned_addr learned my addr 172.21.15.112:0/2794057574 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7e98c32640 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183802ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.148+0000 7f7e99433640 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183821460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e93fff640 1 -- 172.21.15.112:0/2794057574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 msgr2=0x556183821460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e93fff640 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183821460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e93fff640 1 -- 172.21.15.112:0/2794057574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 msgr2=0x556183802ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e93fff640 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183802ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e93fff640 1 -- 172.21.15.112:0/2794057574 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556183809eb0 con 0x55618343e4d0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e99433640 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183821460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e93fff640 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 0x556183820f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009c00 tx=0x7f7e80009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38a093e68a41adc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e7e7fc640 1 -- 172.21.15.112:0/2794057574 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80029030 con 0x55618343e4d0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e7e7fc640 1 -- 172.21.15.112:0/2794057574 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e80005040 con 0x55618343e4d0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.149+0000 7f7e7e7fc640 1 -- 172.21.15.112:0/2794057574 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80025540 con 0x55618343e4d0 2024-04-25T16:50:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 -- 172.21.15.112:0/2794057574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 msgr2=0x556183820f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 0x556183820f20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009c00 tx=0x7f7e80009910 comp rx=0 tx=0).stop 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 -- 172.21.15.112:0/2794057574 shutdown_connections 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183821460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55618343e4d0 0x556183820f20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/2794057574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183802ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 -- 172.21.15.112:0/2794057574 >> 172.21.15.112:0/2794057574 conn(0x556183453300 msgr2=0x5561838027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 -- 172.21.15.112:0/2794057574 shutdown_connections 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 -- 172.21.15.112:0/2794057574 wait complete. 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 -- start start 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.150+0000 7f7ea1e28740 1 Processor -- start 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183813050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183813590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 0x556183813ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ae180 con 0x5561833f31c0 2024-04-25T16:50:48.153 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941af5f0 con 0x5561834362f0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e941ae850 con 0x55618345f1b0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7ea1e28740 10 client.0 _ll_register_callbacks cb 0x7f7e8c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 0x556183813ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 0x556183813ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:37658/0 (socket says 172.21.15.112:37658) 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 -- 172.21.15.112:0/1447241092 learned_addr learned my addr 172.21.15.112:0/1447241092 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e98c32640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183813050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e93fff640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183813590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 -- 172.21.15.112:0/1447241092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 msgr2=0x556183813590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183813590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 -- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 msgr2=0x556183813050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183813050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 -- 172.21.15.112:0/1447241092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556183882ae0 con 0x5561833f31c0 2024-04-25T16:50:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e99433640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 0x556183813ad0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009a10 tx=0x7f7e8c02f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53bef77eee0575e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e73fff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.151+0000 7f7e73fff640 1 -- 172.21.15.112:0/1447241092 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c031860 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7e73fff640 1 -- 172.21.15.112:0/1447241092 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7f7e8c02f660 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e80005040 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7e73fff640 1 -- 172.21.15.112:0/1447241092 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c06b050 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 10 client.16431 fetch_fsmap learned FSMap version 36 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556183883c50 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7e73fff640 1 -- 172.21.15.112:0/1447241092 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e80005040 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7e73fff640 1 -- 172.21.15.112:0/1447241092 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f7e8c04c4e0 con 0x5561833f31c0 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 10 client.16431 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 20 client.16431 populate_metadata read hostname 'smithi112' 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556183884360 0x556183886780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 4 client.16431 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:50:48.155 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55618376cee0 con 0x556183884360 2024-04-25T16:50:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.152+0000 7f7e98c32640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556183884360 0x556183886780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:50:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.153+0000 7f7e73fff640 10 client.16431 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:48.156 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.153+0000 7f7e98c32640 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556183884360 0x556183886780 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20107aa33385804e server_cookie=65b06d95ad14de44 in_seq=0 out_seq=0 2024-04-25T16:50:48.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:48 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:50:48.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:48 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:50:48.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:48 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7e73fff640 1 -- 172.21.15.112:0/1447241092 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55618376cee0 con 0x556183884360 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7e73fff640 10 client.16431 handle_command_reply: tid=0 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 client.16431 shutdown 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 msgr2=0x556183813ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 0x556183813ad0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c009a10 tx=0x7f7e8c02f490 comp rx=0 tx=0).stop 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 shutdown_connections 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556183884360 0x556183886780 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7e73fff640 0 client.16431 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55618345f1b0 0x556183813590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5561833f31c0 0x556183813ad0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.453+0000 7f7ea1e28740 1 --2- 172.21.15.112:0/1447241092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5561834362f0 0x556183813050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.454+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 >> 172.21.15.112:0/1447241092 conn(0x556183453300 msgr2=0x5561837fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.454+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 shutdown_connections 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.454+0000 7f7ea1e28740 1 -- 172.21.15.112:0/1447241092 wait complete. 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.454+0000 7f7ea1e28740 20 client.16431 trim_cache size 0 max 16384 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:50:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e840afcd0 msgr2=0x7f7e840b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e840afcd0 0x7f7e840b21e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c02f500 tx=0x7f7e8c009980 comp rx=0 tx=0).stop 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 msgr2=0x7f7e9419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9419a710 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e9c0 tx=0x7f7e8800ee80 comp rx=0 tx=0).stop 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 shutdown_connections 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f7e840afcd0 0x7f7e840b21e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.454 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e94108a90 0x7f7e9419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e94108f30 0x7f7e9419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:50:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 --2- 172.21.15.112:0/1674545087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e941031e0 0x7f7e9419a710 secure :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800e9c0 tx=0x7f7e8800ee80 comp rx=0 tx=0).stop 2024-04-25T16:50:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 >> 172.21.15.112:0/1674545087 conn(0x7f7e940fe260 msgr2=0x7f7e941066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:50:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 shutdown_connections 2024-04-25T16:50:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:50:48.455+0000 7f7e9ae96640 1 -- 172.21.15.112:0/1674545087 wait complete. 2024-04-25T16:50:48.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:48 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:50:48.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:48 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:50:48.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:48 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:48.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:48 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:50:48.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:48 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:50:48.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:48 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:50:48.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1290694 10.08 MB/sec execute 534 sec latency 92.024 ms 2024-04-25T16:50:49.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:49 smithi133 ceph-mon[32691]: pgmap v393: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-04-25T16:50:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:49 smithi116 ceph-mon[32712]: pgmap v393: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-04-25T16:50:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:49 smithi112 ceph-mon[29122]: pgmap v393: 97 pgs: 97 active+clean; 78 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.1 MiB/s wr, 227 op/s 2024-04-25T16:50:49.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1292781 10.08 MB/sec execute 535 sec latency 16.674 ms 2024-04-25T16:50:50.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1294072 10.07 MB/sec execute 536 sec latency 46.134 ms 2024-04-25T16:50:51.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:51 smithi133 ceph-mon[32691]: pgmap v394: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-04-25T16:50:51.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:51 smithi116 ceph-mon[32712]: pgmap v394: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-04-25T16:50:51.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:51 smithi112 ceph-mon[29122]: pgmap v394: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 391 op/s 2024-04-25T16:50:51.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1296245 10.08 MB/sec execute 537 sec latency 21.988 ms 2024-04-25T16:50:52.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1297240 10.06 MB/sec execute 538 sec latency 18.423 ms 2024-04-25T16:50:53.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:53 smithi133 ceph-mon[32691]: pgmap v395: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2024-04-25T16:50:53.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:53 smithi116 ceph-mon[32712]: pgmap v395: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2024-04-25T16:50:53.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:53 smithi112 ceph-mon[29122]: pgmap v395: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2024-04-25T16:50:53.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1299738 10.08 MB/sec execute 539 sec latency 16.882 ms 2024-04-25T16:50:54.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1300705 10.06 MB/sec execute 540 sec latency 21.081 ms 2024-04-25T16:50:55.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:55 smithi133 ceph-mon[32691]: pgmap v396: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 327 op/s 2024-04-25T16:50:55.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:55 smithi116 ceph-mon[32712]: pgmap v396: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 327 op/s 2024-04-25T16:50:55.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:55 smithi112 ceph-mon[29122]: pgmap v396: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 327 op/s 2024-04-25T16:50:55.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1302739 10.07 MB/sec execute 541 sec latency 58.652 ms 2024-04-25T16:50:56.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1303904 10.06 MB/sec execute 542 sec latency 41.880 ms 2024-04-25T16:50:57.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:57 smithi133 ceph-mon[32691]: pgmap v397: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 251 op/s 2024-04-25T16:50:57.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:57 smithi116 ceph-mon[32712]: pgmap v397: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 251 op/s 2024-04-25T16:50:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:57 smithi112 ceph-mon[29122]: pgmap v397: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 251 op/s 2024-04-25T16:50:57.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1305842 10.05 MB/sec execute 543 sec latency 25.575 ms 2024-04-25T16:50:58.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1307356 10.06 MB/sec execute 544 sec latency 15.672 ms 2024-04-25T16:50:59.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:50:59 smithi133 ceph-mon[32691]: pgmap v398: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 251 op/s 2024-04-25T16:50:59.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:50:59 smithi116 ceph-mon[32712]: pgmap v398: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 251 op/s 2024-04-25T16:50:59.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:50:59 smithi112 ceph-mon[29122]: pgmap v398: 97 pgs: 97 active+clean; 74 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 251 op/s 2024-04-25T16:50:59.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1309343 10.05 MB/sec execute 545 sec latency 23.503 ms 2024-04-25T16:51:00.897 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1310632 10.05 MB/sec execute 546 sec latency 87.556 ms 2024-04-25T16:51:01.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:01 smithi133 ceph-mon[32691]: pgmap v399: 97 pgs: 97 active+clean; 80 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 331 op/s 2024-04-25T16:51:01.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:01 smithi116 ceph-mon[32712]: pgmap v399: 97 pgs: 97 active+clean; 80 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 331 op/s 2024-04-25T16:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/3422592686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108a90 msgr2=0x7fcdb4102630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/3422592686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108a90 0x7fcdb4102630 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009ca0 tx=0x7fcda402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/3422592686 shutdown_connections 2024-04-25T16:51:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/3422592686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 0x7fcdb410f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/3422592686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdb4108f30 0x7fcdb4102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/3422592686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108a90 0x7fcdb4102630 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/3422592686 >> 172.21.15.112:0/3422592686 conn(0x7fcdb40fe260 msgr2=0x7fcdb4100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/3422592686 shutdown_connections 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.580+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/3422592686 wait complete. 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 Processor -- start 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 -- start start 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 0x7fcdb419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdb4108a90 0x7fcdb419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 0x7fcdb419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4112c80 con 0x7fcdb4108f30 2024-04-25T16:51:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4112b00 con 0x7fcdb4108a90 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.581+0000 7fcdb8c9a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4112e00 con 0x7fcdb41031e0 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 0x7fcdb419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 0x7fcdb419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdb4108a90 0x7fcdb419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 0x7fcdb419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39770/0 (socket says 172.21.15.112:39770) 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 0x7fcdb419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:52494/0 (socket says 172.21.15.112:52494) 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 -- 172.21.15.112:0/4156960658 learned_addr learned my addr 172.21.15.112:0/4156960658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 -- 172.21.15.112:0/4156960658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 msgr2=0x7fcdb419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 0x7fcdb419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 -- 172.21.15.112:0/4156960658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdb4108a90 msgr2=0x7fcdb419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdb4108a90 0x7fcdb419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 -- 172.21.15.112:0/4156960658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb419f5d0 con 0x7fcdb4108f30 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb2d9d640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 0x7fcdb419ef90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e970 tx=0x7fcdac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcda37fe640 1 -- 172.21.15.112:0/4156960658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ccc0 con 0x7fcdb4108f30 2024-04-25T16:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcda37fe640 1 -- 172.21.15.112:0/4156960658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdac00ce60 con 0x7fcdb4108f30 2024-04-25T16:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb419f8c0 con 0x7fcdb4108f30 2024-04-25T16:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcda37fe640 1 -- 172.21.15.112:0/4156960658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac010430 con 0x7fcdb4108f30 2024-04-25T16:51:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.582+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb41a7240 con 0x7fcdb4108f30 2024-04-25T16:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 Processor -- start 2024-04-25T16:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcda37fe640 1 -- 172.21.15.112:0/4156960658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fcdac004f90 con 0x7fcdb4108f30 2024-04-25T16:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 -- start start 2024-04-25T16:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b058721af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058722030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b058722570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b058688460 con 0x55b0583614d0 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b058642590 con 0x55b0583592f0 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.585+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b058684420 con 0x55b0583821b0 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058722030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b058722570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058722030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:38298/0 (socket says 172.21.15.112:38298) 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 -- 172.21.15.112:0/1172546290 learned_addr learned my addr 172.21.15.112:0/1172546290 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b058721af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcda37fe640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcd9c0afc50 0x7fcd9c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 -- 172.21.15.112:0/1172546290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 msgr2=0x55b058721af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b058721af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 -- 172.21.15.112:0/1172546290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 msgr2=0x55b058722570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b058722570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 -- 172.21.15.112:0/1172546290 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b05864f0d0 con 0x55b0583592f0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b058722570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb2d9d640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcd9c0afc50 0x7fcd9c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcda37fe640 1 -- 172.21.15.112:0/4156960658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fcdac0df450 con 0x7fcdb4108f30 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058722030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009ca0 tx=0x7fcda40093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86f0ed8ccb35e612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcda0ff9640 1 -- 172.21.15.112:0/1172546290 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4031cf0 con 0x55b0583592f0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcdb2d9d640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcd9c0afc50 0x7fcd9c0b2160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0dcc00 tx=0x7fcdac090040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcda0ff9640 1 -- 172.21.15.112:0/1172546290 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda4036030 con 0x55b0583592f0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.586+0000 7fcda0ff9640 1 -- 172.21.15.112:0/1172546290 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4004040 con 0x55b0583592f0 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1172546290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 msgr2=0x55b058722030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058722030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009ca0 tx=0x7fcda40093f0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1172546290 shutdown_connections 2024-04-25T16:51:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b058722570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058722030 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1172546290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b058721af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1172546290 >> 172.21.15.112:0/1172546290 conn(0x55b058376300 msgr2=0x55b05871e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1172546290 shutdown_connections 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.587+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1172546290 wait complete. 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 -- start start 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 Processor -- start 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583592f0 0x55b05872c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05872c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05873a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b05872ea40 con 0x55b0583614d0 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b05860fb60 con 0x55b0583821b0 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b05872ebc0 con 0x55b0583592f0 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdbfc1f740 10 client.0 _ll_register_callbacks cb 0x55b0586342b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:01.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05872c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05872c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39794/0 (socket says 172.21.15.112:39794) 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdb259c640 1 -- 172.21.15.112:0/1048907347 learned_addr learned my addr 172.21.15.112:0/1048907347 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05873a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.588+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583592f0 0x55b05872c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdb259c640 1 -- 172.21.15.112:0/1048907347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583592f0 msgr2=0x55b05872c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583592f0 0x55b05872c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdb259c640 1 -- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 msgr2=0x55b05873a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05873a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdb259c640 1 -- 172.21.15.112:0/1048907347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b05839a210 con 0x55b0583614d0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05872c6c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005e30 tx=0x7fcda4003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57226677ca657a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcd99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1048907347 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda404a040 con 0x55b0583614d0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1048907347 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fcda4031650 con 0x55b0583614d0 2024-04-25T16:51:01.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda4036030 con 0x55b0583614d0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1048907347 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcda405e050 con 0x55b0583614d0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdbfc1f740 10 client.16443 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b058466620 con 0x55b0583614d0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1048907347 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda4036030 con 0x55b0583614d0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.589+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1048907347 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fcda404d030 con 0x55b0583614d0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdbfc1f740 10 client.16443 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdbfc1f740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdbfc1f740 20 client.16443 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b058340d80 0x55b058746b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdbfc1f740 4 client.16443 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b058696070 con 0x55b058340d80 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b058340d80 0x55b058746b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b058340d80 0x55b058746b50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e249e0723738f134 server_cookie=a0ad43939bacb524 in_seq=0 out_seq=0 2024-04-25T16:51:01.593 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.590+0000 7fcd99ffb640 10 client.16443 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1048907347 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b058696070 con 0x55b058340d80 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcd99ffb640 10 client.16443 handle_command_reply: tid=0 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 client.16443 shutdown 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 msgr2=0x55b05872c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05872c6c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcda4005e30 tx=0x7fcda4003bd0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 shutdown_connections 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05873a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05872c6c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b058340d80 0x55b058746b50 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1048907347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583592f0 0x55b05872c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcd99ffb640 0 client.16443 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 >> 172.21.15.112:0/1048907347 conn(0x55b058376300 msgr2=0x55b058721920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 shutdown_connections 2024-04-25T16:51:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1048907347 wait complete. 2024-04-25T16:51:01.596 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.592+0000 7fcdbfc1f740 20 client.16443 trim_cache size 0 max 16384 2024-04-25T16:51:01.596 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:01 smithi112 ceph-mon[29122]: pgmap v399: 97 pgs: 97 active+clean; 80 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.7 MiB/s wr, 331 op/s 2024-04-25T16:51:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 Processor -- start 2024-04-25T16:51:01.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 -- start start 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 0x55b05873a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583614d0 0x55b058721f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05874c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41b01c0 con 0x55b0583592f0 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ada90 con 0x55b0583821b0 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ab900 con 0x55b0583614d0 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 0x55b05873a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 0x55b05873a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39806/0 (socket says 172.21.15.112:39806) 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 -- 172.21.15.112:0/797708315 learned_addr learned my addr 172.21.15.112:0/797708315 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb259c640 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583614d0 0x55b058721f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05874c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 -- 172.21.15.112:0/797708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583614d0 msgr2=0x55b058721f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583614d0 0x55b058721f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 -- 172.21.15.112:0/797708315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 msgr2=0x55b05874c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05874c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.608+0000 7fcdb1d9b640 1 -- 172.21.15.112:0/797708315 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0587184c0 con 0x55b0583592f0 2024-04-25T16:51:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 0x55b05873a6c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40752f0 tx=0x7fcda80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b4f6b3148fa1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcd9bfff640 1 -- 172.21.15.112:0/797708315 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda802c070 con 0x55b0583592f0 2024-04-25T16:51:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcd9bfff640 1 -- 172.21.15.112:0/797708315 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda80043b0 con 0x55b0583592f0 2024-04-25T16:51:01.608 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcd9bfff640 1 -- 172.21.15.112:0/797708315 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda801a040 con 0x55b0583592f0 2024-04-25T16:51:01.611 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/797708315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 msgr2=0x55b05873a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 0x55b05873a6c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40752f0 tx=0x7fcda80046c0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/797708315 shutdown_connections 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583614d0 0x55b058721f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583821b0 0x55b05874c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.609+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/797708315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583592f0 0x55b05873a6c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/797708315 >> 172.21.15.112:0/797708315 conn(0x55b058376300 msgr2=0x55b058721880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/797708315 shutdown_connections 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/797708315 wait complete. 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- start start 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 Processor -- start 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058740dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.612 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05873e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b05873e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41abf50 con 0x55b0583614d0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ad3c0 con 0x55b0583592f0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41ac620 con 0x55b0583821b0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdbfc1f740 10 client.0 _ll_register_callbacks cb 0x7fcda8009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05873e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdb1d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058740dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdb259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05873e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39814/0 (socket says 172.21.15.112:39814) 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdb259c640 1 -- 172.21.15.112:0/1706728845 learned_addr learned my addr 172.21.15.112:0/1706728845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.610+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b05873e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdb259c640 1 -- 172.21.15.112:0/1706728845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 msgr2=0x55b05873e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b05873e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdb259c640 1 -- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 msgr2=0x55b058740dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:01.613 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058740dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdb259c640 1 -- 172.21.15.112:0/1706728845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b05872cee0 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdb259c640 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05873e180 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcda402a900 tx=0x7fcda4031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99b368b13c9b0c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcd99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1706728845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda40317f0 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda80043b0 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1706728845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8600+0+0 (secure 0 0 0) 0x7fcda4031e00 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1706728845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcda402f660 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 10 client.16449 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b0587228a0 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1706728845 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda80043b0 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1706728845 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7fcda404d040 con 0x55b0583614d0 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 10 client.16449 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:01.614 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 20 client.16449 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.611+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b0583161c0 0x55b0587b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.612+0000 7fcdbfc1f740 4 client.16449 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:51:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.612+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b05868fee0 con 0x55b0583161c0 2024-04-25T16:51:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.612+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b0583161c0 0x55b0587b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.612+0000 7fcdb1d9b640 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b0583161c0 0x55b0587b0d40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa0622b969b020 server_cookie=76fd18ca68da8c55 in_seq=0 out_seq=0 2024-04-25T16:51:01.615 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.612+0000 7fcd99ffb640 10 client.16449 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcd99ffb640 1 -- 172.21.15.112:0/1706728845 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b05868fee0 con 0x55b0583161c0 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcd99ffb640 10 client.16449 handle_command_reply: tid=0 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 client.16449 shutdown 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 msgr2=0x55b05873e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05873e180 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fcda402a900 tx=0x7fcda4031680 comp rx=0 tx=0).stop 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 shutdown_connections 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55b0583161c0 0x55b0587b0d40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b0583821b0 0x55b05873e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.838 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcd99ffb640 0 client.16449 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b0583614d0 0x55b05873e180 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 --2- 172.21.15.112:0/1706728845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0583592f0 0x55b058740dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.839+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 >> 172.21.15.112:0/1706728845 conn(0x55b058376300 msgr2=0x55b05872d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 shutdown_connections 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdbfc1f740 1 -- 172.21.15.112:0/1706728845 wait complete. 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdbfc1f740 20 client.16449 trim_cache size 0 max 16384 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:51:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcd9c0afc50 msgr2=0x7fcd9c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcd9c0afc50 0x7fcd9c0b2160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcdac0dcc00 tx=0x7fcdac090040 comp rx=0 tx=0).stop 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 msgr2=0x7fcdb419ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 0x7fcdb419ef90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e970 tx=0x7fcdac00ed70 comp rx=0 tx=0).stop 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 shutdown_connections 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcd9c0afc50 0x7fcd9c0b2160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcdb4108f30 0x7fcdb419ef90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdb4108a90 0x7fcdb419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 --2- 172.21.15.112:0/4156960658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb41031e0 0x7fcdb419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.840+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 >> 172.21.15.112:0/4156960658 conn(0x7fcdb40fe260 msgr2=0x7fcdb4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.841+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 shutdown_connections 2024-04-25T16:51:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:01.841+0000 7fcdb8c9a640 1 -- 172.21.15.112:0/4156960658 wait complete. 2024-04-25T16:51:01.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1311244 10.04 MB/sec execute 547 sec latency 133.081 ms 2024-04-25T16:51:02.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1313354 10.04 MB/sec execute 548 sec latency 18.542 ms 2024-04-25T16:51:03.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:03 smithi133 ceph-mon[32691]: pgmap v400: 97 pgs: 97 active+clean; 80 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-04-25T16:51:03.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:03 smithi116 ceph-mon[32712]: pgmap v400: 97 pgs: 97 active+clean; 80 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-04-25T16:51:03.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:03 smithi112 ceph-mon[29122]: pgmap v400: 97 pgs: 97 active+clean; 80 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-04-25T16:51:03.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1314555 10.04 MB/sec execute 549 sec latency 19.142 ms 2024-04-25T16:51:04.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1316568 10.03 MB/sec execute 550 sec latency 29.511 ms 2024-04-25T16:51:05.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:05 smithi133 ceph-mon[32691]: pgmap v401: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 338 op/s 2024-04-25T16:51:05.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:05 smithi116 ceph-mon[32712]: pgmap v401: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 338 op/s 2024-04-25T16:51:05.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:05 smithi112 ceph-mon[29122]: pgmap v401: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 338 op/s 2024-04-25T16:51:05.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1317938 10.03 MB/sec execute 551 sec latency 68.435 ms 2024-04-25T16:51:06.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1318459 10.02 MB/sec execute 552 sec latency 49.194 ms 2024-04-25T16:51:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:07 smithi133 ceph-mon[32691]: pgmap v402: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-04-25T16:51:07.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:07 smithi133 ceph-mon[32691]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T16:51:07.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:07 smithi116 ceph-mon[32712]: pgmap v402: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-04-25T16:51:07.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:07 smithi116 ceph-mon[32712]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T16:51:07.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:07 smithi112 ceph-mon[29122]: pgmap v402: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 251 op/s 2024-04-25T16:51:07.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:07 smithi112 ceph-mon[29122]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T16:51:07.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1318999 10.01 MB/sec execute 553 sec latency 75.446 ms 2024-04-25T16:51:08.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1319242 9.99 MB/sec execute 554 sec latency 37.972 ms 2024-04-25T16:51:09.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:09 smithi133 ceph-mon[32691]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 301 op/s 2024-04-25T16:51:09.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:09 smithi116 ceph-mon[32712]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 301 op/s 2024-04-25T16:51:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:09 smithi112 ceph-mon[29122]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.1 MiB/s wr, 301 op/s 2024-04-25T16:51:09.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1321179 9.99 MB/sec execute 555 sec latency 19.389 ms 2024-04-25T16:51:10.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1322513 9.99 MB/sec execute 556 sec latency 21.806 ms 2024-04-25T16:51:11.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:11 smithi133 ceph-mon[32691]: pgmap v405: 97 pgs: 27 active+clean+snaptrim_wait, 60 active+clean, 10 active+clean+snaptrim; 72 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-25T16:51:11.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:11 smithi116 ceph-mon[32712]: pgmap v405: 97 pgs: 27 active+clean+snaptrim_wait, 60 active+clean, 10 active+clean+snaptrim; 72 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-25T16:51:11.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:11 smithi112 ceph-mon[29122]: pgmap v405: 97 pgs: 27 active+clean+snaptrim_wait, 60 active+clean, 10 active+clean+snaptrim; 72 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-25T16:51:11.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1324610 9.99 MB/sec execute 557 sec latency 101.721 ms 2024-04-25T16:51:12.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1325325 9.98 MB/sec execute 558 sec latency 119.892 ms 2024-04-25T16:51:13.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:13 smithi133 ceph-mon[32691]: pgmap v406: 97 pgs: 27 active+clean+snaptrim_wait, 60 active+clean, 10 active+clean+snaptrim; 72 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-25T16:51:13.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:13 smithi116 ceph-mon[32712]: pgmap v406: 97 pgs: 27 active+clean+snaptrim_wait, 60 active+clean, 10 active+clean+snaptrim; 72 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-25T16:51:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:13 smithi112 ceph-mon[29122]: pgmap v406: 97 pgs: 27 active+clean+snaptrim_wait, 60 active+clean, 10 active+clean+snaptrim; 72 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 279 op/s 2024-04-25T16:51:13.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1327244 9.97 MB/sec execute 559 sec latency 82.128 ms 2024-04-25T16:51:14.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1328765 9.97 MB/sec execute 560 sec latency 24.884 ms 2024-04-25T16:51:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 -- 172.21.15.112:0/2799242524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108f00 msgr2=0x7f6ffc102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 --2- 172.21.15.112:0/2799242524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108f00 0x7f6ffc102b50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009900 tx=0x7f6fe402a410 comp rx=0 tx=0).stop 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 -- 172.21.15.112:0/2799242524 shutdown_connections 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 --2- 172.21.15.112:0/2799242524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc1031c0 0x7f6ffc10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 --2- 172.21.15.112:0/2799242524 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108f00 0x7f6ffc102b50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 --2- 172.21.15.112:0/2799242524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc108a60 0x7f6ffc102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 -- 172.21.15.112:0/2799242524 >> 172.21.15.112:0/2799242524 conn(0x7f6ffc0781b0 msgr2=0x7f6ffc0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 -- 172.21.15.112:0/2799242524 shutdown_connections 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.942+0000 7f7001c51640 1 -- 172.21.15.112:0/2799242524 wait complete. 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 Processor -- start 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 -- start start 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc1031c0 0x7f6ffc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 0x7f6ffc19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc108f00 0x7f6ffc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113d80 con 0x7f6ffc108a60 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113c00 con 0x7f6ffc1031c0 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f7001c51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc113f00 con 0x7f6ffc108f00 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 0x7f6ffc19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.943+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 0x7f6ffc19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56998/0 (socket says 172.21.15.112:56998) 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 -- 172.21.15.112:0/4112384375 learned_addr learned my addr 172.21.15.112:0/4112384375 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc1031c0 0x7f6ffc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc108f00 0x7f6ffc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 -- 172.21.15.112:0/4112384375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc108f00 msgr2=0x7f6ffc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc108f00 0x7f6ffc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 -- 172.21.15.112:0/4112384375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc1031c0 msgr2=0x7f6ffc19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc1031c0 0x7f6ffc19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 -- 172.21.15.112:0/4112384375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc19f620 con 0x7f6ffc108a60 2024-04-25T16:51:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc1031c0 0x7f6ffc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 0x7f6ffc19ad20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006010 tx=0x7f6fe4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff97fa640 1 -- 172.21.15.112:0/4112384375 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe403f030 con 0x7f6ffc108a60 2024-04-25T16:51:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff97fa640 1 -- 172.21.15.112:0/4112384375 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe4003aa0 con 0x7f6ffc108a60 2024-04-25T16:51:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc19f8b0 con 0x7f6ffc108a60 2024-04-25T16:51:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f6ff97fa640 1 -- 172.21.15.112:0/4112384375 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4035070 con 0x7f6ffc108a60 2024-04-25T16:51:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.944+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc19fda0 con 0x7f6ffc108a60 2024-04-25T16:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ff97fa640 1 -- 172.21.15.112:0/4112384375 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f6fe4042030 con 0x7f6ffc108a60 2024-04-25T16:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 Processor -- start 2024-04-25T16:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 -- start start 2024-04-25T16:51:14.950 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 0x562b85e2b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 0x562b85e2bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e2c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562b85d92460 con 0x562b85a632f0 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562b85d4c590 con 0x562b85a8c1b0 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f7008be8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562b85d8e420 con 0x562b85a6b4d0 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 0x562b85e2bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 0x562b85e2b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 0x562b85e2bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57006/0 (socket says 172.21.15.112:57006) 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 0x562b85e2b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57918/0 (socket says 172.21.15.112:57918) 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffb7fe640 1 -- 172.21.15.112:0/336266538 learned_addr learned my addr 172.21.15.112:0/336266538 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e2c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ff97fa640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6fe80afc50 0x7f6fe80b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 -- 172.21.15.112:0/336266538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 msgr2=0x562b85e2c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.951 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e2c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 -- 172.21.15.112:0/336266538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 msgr2=0x562b85e2bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 0x562b85e2bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffbfff640 1 -- 172.21.15.112:0/336266538 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b85d590d0 con 0x562b85a8c1b0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.947+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 0x562b85e2bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.948+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6fe80afc50 0x7f6fe80b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.948+0000 7f6ff97fa640 1 -- 172.21.15.112:0/4112384375 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8644+0+0 (secure 0 0 0) 0x7f6fe40ff4f0 con 0x7f6ffc108a60 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.948+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 0x562b85e2b980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400af50 tx=0x7f6ff400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca1cc1292341c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.948+0000 7f6ff1ffb640 1 -- 172.21.15.112:0/336266538 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff401ac60 con 0x562b85a8c1b0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.948+0000 7f6ff1ffb640 1 -- 172.21.15.112:0/336266538 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff401ae00 con 0x562b85a8c1b0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.948+0000 7f6ff1ffb640 1 -- 172.21.15.112:0/336266538 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4004e60 con 0x562b85a8c1b0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 -- 172.21.15.112:0/336266538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 msgr2=0x562b85e2b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6fe80afc50 0x7f6fe80b2160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4020000 tx=0x7f6ff401c030 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:14.952 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 0x562b85e2b980 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400af50 tx=0x7f6ff400f850 comp rx=0 tx=0).stop 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 -- 172.21.15.112:0/336266538 shutdown_connections 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e2c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a632f0 0x562b85e2bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 --2- 172.21.15.112:0/336266538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a8c1b0 0x562b85e2b980 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 -- 172.21.15.112:0/336266538 >> 172.21.15.112:0/336266538 conn(0x562b85a80300 msgr2=0x562b85e282a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 -- 172.21.15.112:0/336266538 shutdown_connections 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 -- 172.21.15.112:0/336266538 wait complete. 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 -- start start 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.949+0000 7f7008be8740 1 Processor -- start 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e3bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 0x562b85e3c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a8c1b0 0x562b85e3a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x562b85e3d090 con 0x562b85a6b4d0 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562b85d19b60 con 0x562b85a632f0 2024-04-25T16:51:14.953 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562b85e3d210 con 0x562b85a8c1b0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 10 client.0 _ll_register_callbacks cb 0x562b85d3e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 0x562b85e3c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 0x562b85e3c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57010/0 (socket says 172.21.15.112:57010) 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e3bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 -- 172.21.15.112:0/2765552640 learned_addr learned my addr 172.21.15.112:0/2765552640 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a8c1b0 0x562b85e3a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 -- 172.21.15.112:0/2765552640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a8c1b0 msgr2=0x562b85e3a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a8c1b0 0x562b85e3a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 -- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 msgr2=0x562b85e3bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e3bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 -- 172.21.15.112:0/2765552640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b85aa4210 con 0x562b85a6b4d0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 0x562b85e3c4a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4031440 tx=0x7f6fe403b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5740f0ab77efc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:14.954 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6fdaffd640 1 -- 172.21.15.112:0/2765552640 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4049070 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f6fdaffd640 1 -- 172.21.15.112:0/2765552640 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8644+0+0 (secure 0 0 0) 0x7f6fe40fce40 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.950+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff401ae00 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f6fdaffd640 1 -- 172.21.15.112:0/2765552640 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6fe40c6e30 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 10 client.16461 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562b85b70620 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f6fdaffd640 1 -- 172.21.15.112:0/2765552640 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff401ae00 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f6fdaffd640 1 -- 172.21.15.112:0/2765552640 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f6fe40c3950 con 0x562b85a6b4d0 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 10 client.16461 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 20 client.16461 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a4ad80 0x562b85e4cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 4 client.16461 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b85da0070 con 0x562b85a4ad80 2024-04-25T16:51:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.951+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a4ad80 0x562b85e4cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.952+0000 7f6fdaffd640 10 client.16461 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.952+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a4ad80 0x562b85e4cb70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecddf3710bbef6c4 server_cookie=4be2cfbeffe01ea6 in_seq=0 out_seq=0 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.954+0000 7f6fdaffd640 1 -- 172.21.15.112:0/2765552640 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b85da0070 con 0x562b85a4ad80 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.954+0000 7f6fdaffd640 10 client.16461 handle_command_reply: tid=0 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.954+0000 7f7008be8740 1 client.16461 shutdown 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 msgr2=0x562b85e3c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 0x562b85e3c4a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4031440 tx=0x7f6fe403b5d0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 shutdown_connections 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a4ad80 0x562b85e4cb70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a8c1b0 0x562b85e3a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a6b4d0 0x562b85e3c4a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 --2- 172.21.15.112:0/2765552640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e3bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 >> 172.21.15.112:0/2765552640 conn(0x562b85a80300 msgr2=0x562b85e2d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f6fdaffd640 0 client.16461 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 shutdown_connections 2024-04-25T16:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 1 -- 172.21.15.112:0/2765552640 wait complete. 2024-04-25T16:51:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.955+0000 7f7008be8740 20 client.16461 trim_cache size 0 max 16384 2024-04-25T16:51:14.970 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 Processor -- start 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 -- start start 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a632f0 0x562b85e52340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a6b4d0 0x562b85e52880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85e2d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1b0270 con 0x562b85a8c1b0 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1adb40 con 0x562b85a6b4d0 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f7008be8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1ab9b0 con 0x562b85a632f0 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85e2d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85e2d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57020/0 (socket says 172.21.15.112:57020) 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffbfff640 1 -- 172.21.15.112:0/2230100610 learned_addr learned my addr 172.21.15.112:0/2230100610 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a6b4d0 0x562b85e52880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.971 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a632f0 0x562b85e52340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffbfff640 1 -- 172.21.15.112:0/2230100610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a632f0 msgr2=0x562b85e52340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a632f0 0x562b85e52340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.972+0000 7f6ffbfff640 1 -- 172.21.15.112:0/2230100610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a6b4d0 msgr2=0x562b85e52880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a6b4d0 0x562b85e52880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ffbfff640 1 -- 172.21.15.112:0/2230100610 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b85e28ef0 con 0x562b85a8c1b0 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a6b4d0 0x562b85e52880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a632f0 0x562b85e52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ffbfff640 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85e2d720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400a540 tx=0x7f6ff4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6f6d73133a3948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ff17fa640 1 -- 172.21.15.112:0/2230100610 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4005a00 con 0x562b85a8c1b0 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ff17fa640 1 -- 172.21.15.112:0/2230100610 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff4012110 con 0x562b85a8c1b0 2024-04-25T16:51:14.972 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.973+0000 7f6ff17fa640 1 -- 172.21.15.112:0/2230100610 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4003bd0 con 0x562b85a8c1b0 2024-04-25T16:51:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 -- 172.21.15.112:0/2230100610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 msgr2=0x562b85e2d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85e2d720 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400a540 tx=0x7f6ff4000f40 comp rx=0 tx=0).stop 2024-04-25T16:51:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 -- 172.21.15.112:0/2230100610 shutdown_connections 2024-04-25T16:51:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85e2d720 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.975 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a6b4d0 0x562b85e52880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 --2- 172.21.15.112:0/2230100610 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a632f0 0x562b85e52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 -- 172.21.15.112:0/2230100610 >> 172.21.15.112:0/2230100610 conn(0x562b85a80300 msgr2=0x562b85e38d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 -- 172.21.15.112:0/2230100610 shutdown_connections 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 -- 172.21.15.112:0/2230100610 wait complete. 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.974+0000 7f7008be8740 1 -- start start 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 Processor -- start 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e40c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85eb3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.976 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1ac000 con 0x562b85a8c1b0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1ad470 con 0x562b85a632f0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc1ac6d0 con 0x562b85a6b4d0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f7008be8740 10 client.0 _ll_register_callbacks cb 0x7f6fe4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f6ffbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85eb3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f6ffb7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57934/0 (socket says 172.21.15.112:57934) 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f6ffb7fe640 1 -- 172.21.15.112:0/3462410030 learned_addr learned my addr 172.21.15.112:0/3462410030 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.975+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e40c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ffb7fe640 1 -- 172.21.15.112:0/3462410030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 msgr2=0x562b85e40c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e40c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ffb7fe640 1 -- 172.21.15.112:0/3462410030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 msgr2=0x562b85eb3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85eb3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ffb7fe640 1 -- 172.21.15.112:0/3462410030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b85eb4440 con 0x562b85a632f0 2024-04-25T16:51:14.977 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e40c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6ffb7fe640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e406d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x562b85e39cb0 tx=0x7f6fec0125c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c1de9a60f6b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6fdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6fdaffd640 1 -- 172.21.15.112:0/3462410030 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec020610 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ff4012110 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6fdaffd640 1 -- 172.21.15.112:0/3462410030 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8644+0+0 (secure 0 0 0) 0x7f6fec0207b0 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6fdaffd640 1 -- 172.21.15.112:0/3462410030 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6fec0209b0 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f7008be8740 10 client.26150 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562b85e2e970 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.976+0000 7f6fdaffd640 1 -- 172.21.15.112:0/3462410030 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff4012110 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f6fdaffd640 1 -- 172.21.15.112:0/3462410030 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f6fec042650 con 0x562b85a632f0 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f7008be8740 10 client.26150 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f7008be8740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f7008be8740 20 client.26150 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:14.978 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f7008be8740 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a201c0 0x562b85e310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f7008be8740 4 client.26150 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:51:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b85e4d0d0 con 0x562b85a201c0 2024-04-25T16:51:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a201c0 0x562b85e310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f6fdaffd640 10 client.26150 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:14.979 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:14.977+0000 7f6ff3fff640 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a201c0 0x562b85e310e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4f45e6f5b0ce08 server_cookie=5325c725ddd55edf in_seq=0 out_seq=0 2024-04-25T16:51:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.149+0000 7f6fdaffd640 1 -- 172.21.15.112:0/3462410030 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b85e4d0d0 con 0x562b85a201c0 2024-04-25T16:51:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.149+0000 7f6fdaffd640 10 client.26150 handle_command_reply: tid=0 2024-04-25T16:51:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 client.26150 shutdown 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 msgr2=0x562b85e406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e406d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x562b85e39cb0 tx=0x7f6fec0125c0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 shutdown_connections 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x562b85a201c0 0x562b85e310e0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562b85a6b4d0 0x562b85e40c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x562b85a8c1b0 0x562b85eb3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f6fdaffd640 0 client.26150 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 --2- 172.21.15.112:0/3462410030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562b85a632f0 0x562b85e406d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 >> 172.21.15.112:0/3462410030 conn(0x562b85a80300 msgr2=0x562b85e2d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 shutdown_connections 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 1 -- 172.21.15.112:0/3462410030 wait complete. 2024-04-25T16:51:15.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.150+0000 7f7008be8740 20 client.26150 trim_cache size 0 max 16384 2024-04-25T16:51:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:51:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:51:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:51:15.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6fe80afc50 msgr2=0x7f6fe80b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6fe80afc50 0x7f6fe80b2160 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4020000 tx=0x7f6ff401c030 comp rx=0 tx=0).stop 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 msgr2=0x7f6ffc19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 0x7f6ffc19ad20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4006010 tx=0x7f6fe4004060 comp rx=0 tx=0).stop 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 shutdown_connections 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6fe80afc50 0x7f6fe80b2160 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6ffc108f00 0x7f6ffc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ffc108a60 0x7f6ffc19ad20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 --2- 172.21.15.112:0/4112384375 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ffc1031c0 0x7f6ffc19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 >> 172.21.15.112:0/4112384375 conn(0x7f6ffc0781b0 msgr2=0x7f6ffc104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 shutdown_connections 2024-04-25T16:51:15.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:15.151+0000 7f7001c51640 1 -- 172.21.15.112:0/4112384375 wait complete. 2024-04-25T16:51:15.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:15 smithi133 ceph-mon[32691]: pgmap v407: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:15.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:15 smithi116 ceph-mon[32712]: pgmap v407: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:15 smithi112 ceph-mon[29122]: pgmap v407: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:15.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1330321 9.97 MB/sec execute 561 sec latency 67.157 ms 2024-04-25T16:51:16.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1331884 9.97 MB/sec execute 562 sec latency 36.961 ms 2024-04-25T16:51:17.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:17 smithi133 ceph-mon[32691]: pgmap v408: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:17.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:17 smithi133 ceph-mon[32691]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T16:51:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:17 smithi116 ceph-mon[32712]: pgmap v408: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:17.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:17 smithi116 ceph-mon[32712]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T16:51:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:17 smithi112 ceph-mon[29122]: pgmap v408: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:17 smithi112 ceph-mon[29122]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T16:51:17.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1333290 9.96 MB/sec execute 563 sec latency 16.536 ms 2024-04-25T16:51:18.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1335356 9.97 MB/sec execute 564 sec latency 20.239 ms 2024-04-25T16:51:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:19 smithi116 ceph-mon[32712]: pgmap v410: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:19 smithi112 ceph-mon[29122]: pgmap v410: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:19.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:19 smithi133 ceph-mon[32691]: pgmap v410: 97 pgs: 97 active+clean; 77 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 163 op/s 2024-04-25T16:51:19.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1336379 9.96 MB/sec execute 565 sec latency 19.469 ms 2024-04-25T16:51:20.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1338777 9.97 MB/sec execute 566 sec latency 27.293 ms 2024-04-25T16:51:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:21 smithi116 ceph-mon[32712]: pgmap v411: 97 pgs: 97 active+clean; 73 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-04-25T16:51:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:21 smithi112 ceph-mon[29122]: pgmap v411: 97 pgs: 97 active+clean; 73 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-04-25T16:51:21.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:21 smithi133 ceph-mon[32691]: pgmap v411: 97 pgs: 97 active+clean; 73 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-04-25T16:51:21.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1339517 9.95 MB/sec execute 567 sec latency 109.832 ms 2024-04-25T16:51:22.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1341739 9.96 MB/sec execute 568 sec latency 19.018 ms 2024-04-25T16:51:23.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:23 smithi116 ceph-mon[32712]: pgmap v412: 97 pgs: 97 active+clean; 73 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-04-25T16:51:23.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:23 smithi112 ceph-mon[29122]: pgmap v412: 97 pgs: 97 active+clean; 73 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-04-25T16:51:23.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:23 smithi133 ceph-mon[32691]: pgmap v412: 97 pgs: 97 active+clean; 73 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 266 op/s 2024-04-25T16:51:23.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1342932 9.95 MB/sec execute 569 sec latency 17.628 ms 2024-04-25T16:51:24.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1344841 9.95 MB/sec execute 570 sec latency 32.892 ms 2024-04-25T16:51:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:25 smithi116 ceph-mon[32712]: pgmap v413: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-04-25T16:51:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:25 smithi112 ceph-mon[29122]: pgmap v413: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-04-25T16:51:25.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:25 smithi133 ceph-mon[32691]: pgmap v413: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-04-25T16:51:25.898 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1346318 9.95 MB/sec execute 571 sec latency 18.105 ms 2024-04-25T16:51:26.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1347559 9.94 MB/sec execute 572 sec latency 98.649 ms 2024-04-25T16:51:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:27 smithi116 ceph-mon[32712]: pgmap v414: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-04-25T16:51:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:27 smithi112 ceph-mon[29122]: pgmap v414: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-04-25T16:51:27.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:27 smithi133 ceph-mon[32691]: pgmap v414: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 265 op/s 2024-04-25T16:51:27.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1349572 9.95 MB/sec execute 573 sec latency 25.637 ms 2024-04-25T16:51:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 -- 172.21.15.112:0/1924651337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 msgr2=0x7f6c4c102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 --2- 172.21.15.112:0/1924651337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c102b60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009900 tx=0x7f6c4002a410 comp rx=0 tx=0).stop 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 -- 172.21.15.112:0/1924651337 shutdown_connections 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 --2- 172.21.15.112:0/1924651337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c1031d0 0x7f6c4c10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 --2- 172.21.15.112:0/1924651337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c102b60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 --2- 172.21.15.112:0/1924651337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c4c108a80 0x7f6c4c102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 -- 172.21.15.112:0/1924651337 >> 172.21.15.112:0/1924651337 conn(0x7f6c4c079170 msgr2=0x7f6c4c0795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 -- 172.21.15.112:0/1924651337 shutdown_connections 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.255+0000 7f6c521d7640 1 -- 172.21.15.112:0/1924651337 wait complete. 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 Processor -- start 2024-04-25T16:51:28.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 -- start start 2024-04-25T16:51:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c4c1031d0 0x7f6c4c19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c108a80 0x7f6c4c19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c1a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112c70 con 0x7f6c4c108f20 2024-04-25T16:51:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112af0 con 0x7f6c4c108a80 2024-04-25T16:51:28.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c521d7640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112df0 con 0x7f6c4c1031d0 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c1a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c4bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c1a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47372/0 (socket says 172.21.15.112:47372) 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c4bfff640 1 -- 172.21.15.112:0/2208712639 learned_addr learned my addr 172.21.15.112:0/2208712639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.256+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c108a80 0x7f6c4c19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c4c1031d0 0x7f6c4c19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4bfff640 1 -- 172.21.15.112:0/2208712639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c4c1031d0 msgr2=0x7f6c4c19b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4bfff640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c4c1031d0 0x7f6c4c19b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4bfff640 1 -- 172.21.15.112:0/2208712639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c108a80 msgr2=0x7f6c4c19baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4bfff640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c108a80 0x7f6c4c19baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4bfff640 1 -- 172.21.15.112:0/2208712639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c19c370 con 0x7f6c4c108f20 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c108a80 0x7f6c4c19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c4bfff640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c1a6630 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400eb10 tx=0x7f6c4400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c48ff9640 1 -- 172.21.15.112:0/2208712639 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44004b20 con 0x7f6c4c108f20 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c48ff9640 1 -- 172.21.15.112:0/2208712639 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c44004cc0 con 0x7f6c4c108f20 2024-04-25T16:51:28.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1a6c30 con 0x7f6c4c108f20 2024-04-25T16:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c48ff9640 1 -- 172.21.15.112:0/2208712639 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44010670 con 0x7f6c4c108f20 2024-04-25T16:51:28.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.257+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1a7100 con 0x7f6c4c108f20 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.260+0000 7f6c59165740 1 Processor -- start 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.260+0000 7f6c48ff9640 1 -- 172.21.15.112:0/2208712639 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f6c44005020 con 0x7f6c4c108f20 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.260+0000 7f6c59165740 1 -- start start 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c59165740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14222aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c59165740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 0x55a14222b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c59165740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e6a4d0 0x55a14222b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c59165740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a142191460 con 0x55a141e622f0 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c59165740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a14214b590 con 0x55a141e6a4d0 2024-04-25T16:51:28.264 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c59165740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a14218d420 con 0x55a141e8b1b0 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 0x55a14222b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14222aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14222aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:60508/0 (socket says 172.21.15.112:60508) 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 0x55a14222b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47378/0 (socket says 172.21.15.112:47378) 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 -- 172.21.15.112:0/4263829479 learned_addr learned my addr 172.21.15.112:0/4263829479 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e6a4d0 0x55a14222b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c48ff9640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6c340afc50 0x7f6c340b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 -- 172.21.15.112:0/4263829479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e6a4d0 msgr2=0x55a14222b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.265 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e6a4d0 0x55a14222b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 -- 172.21.15.112:0/4263829479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 msgr2=0x55a14222b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 0x55a14222b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 -- 172.21.15.112:0/4263829479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1421580d0 con 0x55a141e8b1b0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 0x55a14222b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4bfff640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6c340afc50 0x7f6c340b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.261+0000 7f6c4affd640 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14222aaf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c400057c0 tx=0x7f6c40000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6f2f25874589d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c48ff9640 1 -- 172.21.15.112:0/2208712639 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f6c440df670 con 0x7f6c4c108f20 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c31ffb640 1 -- 172.21.15.112:0/4263829479 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40035070 con 0x55a141e8b1b0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c31ffb640 1 -- 172.21.15.112:0/4263829479 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400054d0 con 0x55a141e8b1b0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c31ffb640 1 -- 172.21.15.112:0/4263829479 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40004bd0 con 0x55a141e8b1b0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c59165740 1 -- 172.21.15.112:0/4263829479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 msgr2=0x55a14222aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c4bfff640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6c340afc50 0x7f6c340b2160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c440a3840 tx=0x7f6c44090040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.262+0000 7f6c59165740 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14222aaf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c400057c0 tx=0x7f6c40000bc0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.266 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 -- 172.21.15.112:0/4263829479 shutdown_connections 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e6a4d0 0x55a14222b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e622f0 0x55a14222b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 --2- 172.21.15.112:0/4263829479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14222aaf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 -- 172.21.15.112:0/4263829479 >> 172.21.15.112:0/4263829479 conn(0x55a141e7f300 msgr2=0x55a1422272a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 -- 172.21.15.112:0/4263829479 shutdown_connections 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 -- 172.21.15.112:0/4263829479 wait complete. 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 -- start start 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.263+0000 7f6c59165740 1 Processor -- start 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 0x55a1422351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142235720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14223b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.267 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55a142237bc0 con 0x55a141e6a4d0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a142118b60 con 0x55a141e622f0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a142237d40 con 0x55a141e8b1b0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 10 client.0 _ll_register_callbacks cb 0x55a14213d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 0x55a1422351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142235720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 0x55a1422351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57988/0 (socket says 172.21.15.112:57988) 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142235720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47382/0 (socket says 172.21.15.112:47382) 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 -- 172.21.15.112:0/2667291464 learned_addr learned my addr 172.21.15.112:0/2667291464 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14223b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 -- 172.21.15.112:0/2667291464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 msgr2=0x55a14223b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14223b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 -- 172.21.15.112:0/2667291464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 msgr2=0x55a142235720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142235720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 -- 172.21.15.112:0/2667291464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a141ea3210 con 0x55a141e622f0 2024-04-25T16:51:28.268 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142235720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 0x55a1422351e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009320 tx=0x7f6c40041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffeb15cbc61f899d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c2affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c2affd640 1 -- 172.21.15.112:0/2667291464 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40035030 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c400054d0 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.264+0000 7f6c2affd640 1 -- 172.21.15.112:0/2667291464 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f6c400052d0 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c2affd640 1 -- 172.21.15.112:0/2667291464 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6c40004980 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 10 client.26162 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a141f6f620 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c2affd640 1 -- 172.21.15.112:0/2667291464 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c400054d0 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c2affd640 1 -- 172.21.15.112:0/2667291464 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f6c4003dbb0 con 0x55a141e622f0 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 10 client.26162 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 10 client.26162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 20 client.26162 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e49d80 0x55a1422473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 4 client.26162 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a14219f070 con 0x55a141e49d80 2024-04-25T16:51:28.269 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e49d80 0x55a1422473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c2affd640 10 client.26162 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:28.270 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.265+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e49d80 0x55a1422473c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e4617631e52e75 server_cookie=b940568b78e53209 in_seq=0 out_seq=0 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.267+0000 7f6c2affd640 1 -- 172.21.15.112:0/2667291464 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a14219f070 con 0x55a141e49d80 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.267+0000 7f6c2affd640 10 client.26162 handle_command_reply: tid=0 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.267+0000 7f6c59165740 1 client.26162 shutdown 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 msgr2=0x55a1422351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 0x55a1422351e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009320 tx=0x7f6c40041040 comp rx=0 tx=0).stop 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 shutdown_connections 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e49d80 0x55a1422473c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a14223b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142235720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c2affd640 0 client.26162 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 --2- 172.21.15.112:0/2667291464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e622f0 0x55a1422351e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 >> 172.21.15.112:0/2667291464 conn(0x55a141e7f300 msgr2=0x55a14222aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 shutdown_connections 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 1 -- 172.21.15.112:0/2667291464 wait complete. 2024-04-25T16:51:28.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.268+0000 7f6c59165740 20 client.26162 trim_cache size 0 max 16384 2024-04-25T16:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.284+0000 7f6c59165740 1 Processor -- start 2024-04-25T16:51:28.283 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.284+0000 7f6c59165740 1 -- start start 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c59165740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a14223afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c59165740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a14222af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c59165740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142232210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c59165740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1b2440 con 0x55a141e6a4d0 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c59165740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1afd10 con 0x55a141e49d80 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c59165740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1adb80 con 0x55a141e8b1b0 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a14222af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a14223afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a14223afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:57996/0 (socket says 172.21.15.112:57996) 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 -- 172.21.15.112:0/2836113017 learned_addr learned my addr 172.21.15.112:0/2836113017 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a14222af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47396/0 (socket says 172.21.15.112:47396) 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142232210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 -- 172.21.15.112:0/2836113017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 msgr2=0x55a142232210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142232210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 -- 172.21.15.112:0/2836113017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 msgr2=0x55a14222af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a14222af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 -- 172.21.15.112:0/2836113017 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a142236490 con 0x55a141e49d80 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a14222af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.285+0000 7f6c4affd640 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a14223afe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c40036040 tx=0x7f6c40031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283cd5c742fb4d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.286+0000 7f6c317fa640 1 -- 172.21.15.112:0/2836113017 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c40005230 con 0x55a141e49d80 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.286+0000 7f6c317fa640 1 -- 172.21.15.112:0/2836113017 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4004f070 con 0x55a141e49d80 2024-04-25T16:51:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.286+0000 7f6c317fa640 1 -- 172.21.15.112:0/2836113017 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4004ad30 con 0x55a141e49d80 2024-04-25T16:51:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 -- 172.21.15.112:0/2836113017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 msgr2=0x55a14223afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a14223afe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6c40036040 tx=0x7f6c40031400 comp rx=0 tx=0).stop 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 -- 172.21.15.112:0/2836113017 shutdown_connections 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142232210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a14222af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 --2- 172.21.15.112:0/2836113017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a14223afe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 -- 172.21.15.112:0/2836113017 >> 172.21.15.112:0/2836113017 conn(0x55a141e7f300 msgr2=0x55a14222a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 -- 172.21.15.112:0/2836113017 shutdown_connections 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 -- 172.21.15.112:0/2836113017 wait complete. 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 -- start start 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.287+0000 7f6c59165740 1 Processor -- start 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a142240330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142240870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142240db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1ae1d0 con 0x55a141e6a4d0 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1af640 con 0x55a141e49d80 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1ae8a0 con 0x55a141e8b1b0 2024-04-25T16:51:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c59165740 10 client.0 _ll_register_callbacks cb 0x7f6c3c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142240870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a142240330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142240870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:47408/0 (socket says 172.21.15.112:47408) 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 -- 172.21.15.112:0/1556918747 learned_addr learned my addr 172.21.15.112:0/1556918747 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4affd640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142240db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 -- 172.21.15.112:0/1556918747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 msgr2=0x55a142240db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142240db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 -- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 msgr2=0x55a142240330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a142240330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 -- 172.21.15.112:0/1556918747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1422ad6f0 con 0x55a141e6a4d0 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4affd640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a142240330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c4b7fe640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142240870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55a142228230 tx=0x7f6c3c020c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb45027b523b6c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.288+0000 7f6c2affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c2affd640 1 -- 172.21.15.112:0/1556918747 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c030060 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c2affd640 1 -- 172.21.15.112:0/1556918747 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7f6c3c012d60 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c4004f070 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c2affd640 1 -- 172.21.15.112:0/1556918747 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6c3c02b480 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 10 client.16485 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a1422ae9a0 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c2affd640 1 -- 172.21.15.112:0/1556918747 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c4004f070 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c2affd640 1 -- 172.21.15.112:0/1556918747 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f6c3c010430 con 0x55a141e6a4d0 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 10 client.16485 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 20 client.16485 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e1f1c0 0x55a1422b1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 4 client.16485 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:51:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a142230bb0 con 0x55a141e1f1c0 2024-04-25T16:51:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.289+0000 7f6c4affd640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e1f1c0 0x55a1422b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.290+0000 7f6c2affd640 10 client.16485 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.290+0000 7f6c4affd640 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e1f1c0 0x55a1422b1140 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b58fd50915b2120 server_cookie=be405eaeef11bbed in_seq=0 out_seq=0 2024-04-25T16:51:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.644+0000 7f6c2affd640 1 -- 172.21.15.112:0/1556918747 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a142230bb0 con 0x55a141e1f1c0 2024-04-25T16:51:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.644+0000 7f6c2affd640 10 client.16485 handle_command_reply: tid=0 2024-04-25T16:51:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.644+0000 7f6c59165740 1 client.16485 shutdown 2024-04-25T16:51:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 msgr2=0x55a142240870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142240870 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x55a142228230 tx=0x7f6c3c020c50 comp rx=0 tx=0).stop 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 shutdown_connections 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55a141e1f1c0 0x55a1422b1140 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c2affd640 0 client.16485 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a141e8b1b0 0x55a142240db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55a141e6a4d0 0x55a142240870 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 --2- 172.21.15.112:0/1556918747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a141e49d80 0x55a142240330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 >> 172.21.15.112:0/1556918747 conn(0x55a141e7f300 msgr2=0x55a142233770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 shutdown_connections 2024-04-25T16:51:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 1 -- 172.21.15.112:0/1556918747 wait complete. 2024-04-25T16:51:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.645+0000 7f6c59165740 20 client.16485 trim_cache size 0 max 16384 2024-04-25T16:51:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:51:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:51:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:51:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:51:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6c340afc50 msgr2=0x7f6c340b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6c340afc50 0x7f6c340b2160 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6c440a3840 tx=0x7f6c44090040 comp rx=0 tx=0).stop 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 msgr2=0x7f6c4c1a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c1a6630 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400eb10 tx=0x7f6c4400c2d0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 shutdown_connections 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f6c340afc50 0x7f6c340b2160 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c4c108f20 0x7f6c4c1a6630 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c4c108a80 0x7f6c4c19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 --2- 172.21.15.112:0/2208712639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c4c1031d0 0x7f6c4c19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 >> 172.21.15.112:0/2208712639 conn(0x7f6c4c079170 msgr2=0x7f6c4c101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 shutdown_connections 2024-04-25T16:51:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:28.646+0000 7f6c521d7640 1 -- 172.21.15.112:0/2208712639 wait complete. 2024-04-25T16:51:28.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1350608 9.93 MB/sec execute 574 sec latency 82.167 ms 2024-04-25T16:51:29.499 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:29 smithi133 ceph-mon[32691]: pgmap v415: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-04-25T16:51:29.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:29 smithi116 ceph-mon[32712]: pgmap v415: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-04-25T16:51:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:29 smithi112 ceph-mon[29122]: pgmap v415: 97 pgs: 97 active+clean; 79 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 251 op/s 2024-04-25T16:51:29.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1352996 9.94 MB/sec execute 575 sec latency 20.669 ms 2024-04-25T16:51:30.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1353980 9.93 MB/sec execute 576 sec latency 22.372 ms 2024-04-25T16:51:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:31 smithi116 ceph-mon[32712]: pgmap v416: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 374 op/s 2024-04-25T16:51:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:31 smithi112 ceph-mon[29122]: pgmap v416: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 374 op/s 2024-04-25T16:51:31.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:31 smithi133 ceph-mon[32691]: pgmap v416: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 374 op/s 2024-04-25T16:51:31.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1356295 9.94 MB/sec execute 577 sec latency 76.287 ms 2024-04-25T16:51:32.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1357285 9.93 MB/sec execute 578 sec latency 16.681 ms 2024-04-25T16:51:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:33 smithi116 ceph-mon[32712]: pgmap v417: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-04-25T16:51:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:33 smithi112 ceph-mon[29122]: pgmap v417: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-04-25T16:51:33.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:33 smithi133 ceph-mon[32691]: pgmap v417: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-04-25T16:51:33.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1359530 9.93 MB/sec execute 579 sec latency 21.644 ms 2024-04-25T16:51:34.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1360825 9.93 MB/sec execute 580 sec latency 18.992 ms 2024-04-25T16:51:35.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:35 smithi116 ceph-mon[32712]: pgmap v418: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-04-25T16:51:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:35 smithi112 ceph-mon[29122]: pgmap v418: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-04-25T16:51:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:35 smithi133 ceph-mon[32691]: pgmap v418: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-04-25T16:51:35.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1363064 9.93 MB/sec execute 581 sec latency 21.914 ms 2024-04-25T16:51:36.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1364175 9.93 MB/sec execute 582 sec latency 32.507 ms 2024-04-25T16:51:37.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:37 smithi116 ceph-mon[32712]: pgmap v419: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-04-25T16:51:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:37 smithi112 ceph-mon[29122]: pgmap v419: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-04-25T16:51:37.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:37 smithi133 ceph-mon[32691]: pgmap v419: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-04-25T16:51:37.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1365409 9.92 MB/sec execute 583 sec latency 26.982 ms 2024-04-25T16:51:38.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1367443 9.92 MB/sec execute 584 sec latency 20.121 ms 2024-04-25T16:51:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:39 smithi116 ceph-mon[32712]: pgmap v420: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-04-25T16:51:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:39 smithi112 ceph-mon[29122]: pgmap v420: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-04-25T16:51:39.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:39 smithi133 ceph-mon[32691]: pgmap v420: 97 pgs: 97 active+clean; 74 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.4 MiB/s wr, 234 op/s 2024-04-25T16:51:39.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1369044 9.92 MB/sec execute 585 sec latency 21.973 ms 2024-04-25T16:51:40.899 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1370885 9.92 MB/sec execute 586 sec latency 33.243 ms 2024-04-25T16:51:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:41 smithi116 ceph-mon[32712]: pgmap v421: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-04-25T16:51:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:41 smithi112 ceph-mon[29122]: pgmap v421: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-04-25T16:51:41.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:41 smithi133 ceph-mon[32691]: pgmap v421: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.5 MiB/s wr, 312 op/s 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 -- 172.21.15.112:0/867240748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108f00 msgr2=0x7ff13c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 --2- 172.21.15.112:0/867240748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108f00 0x7ff13c102b50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff130009900 tx=0x7ff13002a410 comp rx=0 tx=0).stop 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 -- 172.21.15.112:0/867240748 shutdown_connections 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 --2- 172.21.15.112:0/867240748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 0x7ff13c10f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 --2- 172.21.15.112:0/867240748 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108f00 0x7ff13c102b50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 --2- 172.21.15.112:0/867240748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff13c108a60 0x7ff13c102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 -- 172.21.15.112:0/867240748 >> 172.21.15.112:0/867240748 conn(0x7ff13c0781b0 msgr2=0x7ff13c0785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 -- 172.21.15.112:0/867240748 shutdown_connections 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.770+0000 7ff1420f9640 1 -- 172.21.15.112:0/867240748 wait complete. 2024-04-25T16:51:41.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.771+0000 7ff1420f9640 1 Processor -- start 2024-04-25T16:51:41.770 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.771+0000 7ff1420f9640 1 -- start start 2024-04-25T16:51:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 0x7ff13c1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.771 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 0x7ff13c1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff13c108f00 0x7ff13c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112c80 con 0x7ff13c108a60 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112b00 con 0x7ff13c1031c0 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c112e00 con 0x7ff13c108f00 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 0x7ff13c1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 0x7ff13c1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52002/0 (socket says 172.21.15.112:52002) 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 0x7ff13c1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 -- 172.21.15.112:0/552221984 learned_addr learned my addr 172.21.15.112:0/552221984 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:41.772 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13bfff640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff13c108f00 0x7ff13c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 -- 172.21.15.112:0/552221984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff13c108f00 msgr2=0x7ff13c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff13c108f00 0x7ff13c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 -- 172.21.15.112:0/552221984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 msgr2=0x7ff13c1a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 0x7ff13c1a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 -- 172.21.15.112:0/552221984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c19adc0 con 0x7ff13c108a60 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 0x7ff13c1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff13affd640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 0x7ff13c1a0a50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff130006010 tx=0x7ff130004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff138ff9640 1 -- 172.21.15.112:0/552221984 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13003f030 con 0x7ff13c108a60 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c19aff0 con 0x7ff13c108a60 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.772+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c1a72b0 con 0x7ff13c108a60 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.773+0000 7ff138ff9640 1 -- 172.21.15.112:0/552221984 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff130003aa0 con 0x7ff13c108a60 2024-04-25T16:51:41.773 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.773+0000 7ff138ff9640 1 -- 172.21.15.112:0/552221984 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130049060 con 0x7ff13c108a60 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 Processor -- start 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 -- start start 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff138ff9640 1 -- 172.21.15.112:0/552221984 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7ff130042030 con 0x7ff13c108a60 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be3923a1b0 0x55be395d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 0x55be395da030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395da570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be39540460 con 0x55be392194d0 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55be394fa590 con 0x55be392112f0 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff148faa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55be3953c420 con 0x55be3923a1b0 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff13affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395da570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.775+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 0x55be395da030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.781 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 0x55be395da030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:36058/0 (socket says 172.21.15.112:36058) 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 -- 172.21.15.112:0/619616266 learned_addr learned my addr 172.21.15.112:0/619616266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13bfff640 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be3923a1b0 0x55be395d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff138ff9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff1240afc50 0x7ff1240b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 -- 172.21.15.112:0/619616266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be3923a1b0 msgr2=0x55be395d9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be3923a1b0 0x55be395d9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 -- 172.21.15.112:0/619616266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 msgr2=0x55be395da570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395da570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 -- 172.21.15.112:0/619616266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be395070d0 con 0x55be392112f0 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13bfff640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff1240afc50 0x7ff1240b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff138ff9640 1 -- 172.21.15.112:0/552221984 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff1300ff270 con 0x7ff13c108a60 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 0x55be395da030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00aef0 tx=0x7ff12c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff233688c53b4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:41.782 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff121ffb640 1 -- 172.21.15.112:0/619616266 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c01d070 con 0x55be392112f0 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff121ffb640 1 -- 172.21.15.112:0/619616266 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c010cf0 con 0x55be392112f0 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.776+0000 7ff121ffb640 1 -- 172.21.15.112:0/619616266 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c018d40 con 0x55be392112f0 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 -- 172.21.15.112:0/619616266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 msgr2=0x55be395da030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff13bfff640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff1240afc50 0x7ff1240b2160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55be395d6c60 tx=0x7ff134004a40 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 0x55be395da030 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00aef0 tx=0x7ff12c00f790 comp rx=0 tx=0).stop 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 -- 172.21.15.112:0/619616266 shutdown_connections 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395da570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392112f0 0x55be395da030 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 --2- 172.21.15.112:0/619616266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be3923a1b0 0x55be395d9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 -- 172.21.15.112:0/619616266 >> 172.21.15.112:0/619616266 conn(0x55be3922e300 msgr2=0x55be395d62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 -- 172.21.15.112:0/619616266 shutdown_connections 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.777+0000 7ff148faa740 1 -- 172.21.15.112:0/619616266 wait complete. 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 -- start start 2024-04-25T16:51:41.783 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 Processor -- start 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392194d0 0x55be395e4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be3923a1b0 0x55be395ea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55be395e6c20 con 0x55be392112f0 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55be394c7b60 con 0x55be3923a1b0 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55be395e6da0 con 0x55be392194d0 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff148faa740 10 client.0 _ll_register_callbacks cb 0x55be394ec2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff13b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395e41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52024/0 (socket says 172.21.15.112:52024) 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.778+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3762735370 learned_addr learned my addr 172.21.15.112:0/3762735370 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13bfff640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be3923a1b0 0x55be395ea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.784 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13affd640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392194d0 0x55be395e4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3762735370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392194d0 msgr2=0x55be395e4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392194d0 0x55be395e4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be3923a1b0 msgr2=0x55be395ea180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be3923a1b0 0x55be395ea180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13b7fe640 1 -- 172.21.15.112:0/3762735370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be39252210 con 0x55be392112f0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395e41e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00aaf0 tx=0x7ff12c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590ed500d54a773d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff11affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff11affd640 1 -- 172.21.15.112:0/3762735370 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c01d050 con 0x55be392112f0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff12c010cf0 con 0x55be392112f0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff11affd640 1 -- 172.21.15.112:0/3762735370 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff12c010a50 con 0x55be392112f0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff11affd640 1 -- 172.21.15.112:0/3762735370 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff12c005040 con 0x55be392112f0 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff148faa740 10 client.16497 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:41.785 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.779+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be3931e620 con 0x55be392112f0 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff11affd640 1 -- 172.21.15.112:0/3762735370 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c010cf0 con 0x55be392112f0 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff11affd640 1 -- 172.21.15.112:0/3762735370 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7ff12c004a80 con 0x55be392112f0 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff148faa740 10 client.16497 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff148faa740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff148faa740 20 client.16497 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff148faa740 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be391f8d80 0x55be395f66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff148faa740 4 client.16497 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3954e070 con 0x55be391f8d80 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff13affd640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be391f8d80 0x55be395f66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff11affd640 10 client.16497 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.780+0000 7ff13affd640 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be391f8d80 0x55be395f66b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33bd48d4d1352fa server_cookie=a5e6d1ad92071c04 in_seq=0 out_seq=0 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff11affd640 1 -- 172.21.15.112:0/3762735370 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be3954e070 con 0x55be391f8d80 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff11affd640 10 client.16497 handle_command_reply: tid=0 2024-04-25T16:51:41.786 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 client.16497 shutdown 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 msgr2=0x55be395e41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395e41e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00aaf0 tx=0x7ff12c010f10 comp rx=0 tx=0).stop 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 shutdown_connections 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be391f8d80 0x55be395f66b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392194d0 0x55be395e4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff11affd640 0 client.16497 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be3923a1b0 0x55be395ea180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 --2- 172.21.15.112:0/3762735370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395e41e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 >> 172.21.15.112:0/3762735370 conn(0x55be3922e300 msgr2=0x55be395d9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 shutdown_connections 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 1 -- 172.21.15.112:0/3762735370 wait complete. 2024-04-25T16:51:41.787 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.782+0000 7ff148faa740 20 client.16497 trim_cache size 0 max 16384 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 Processor -- start 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 -- start start 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be391f8d80 0x55be395dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392112f0 0x55be395ea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395e1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1b24a0 con 0x55be392194d0 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1afd70 con 0x55be391f8d80 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff148faa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1adbe0 con 0x55be392112f0 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52034/0 (socket says 172.21.15.112:52034) 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 -- 172.21.15.112:0/2569352171 learned_addr learned my addr 172.21.15.112:0/2569352171 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13affd640 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392112f0 0x55be395ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be391f8d80 0x55be395dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 -- 172.21.15.112:0/2569352171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392112f0 msgr2=0x55be395ea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392112f0 0x55be395ea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 -- 172.21.15.112:0/2569352171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be391f8d80 msgr2=0x55be395dddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be391f8d80 0x55be395dddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13bfff640 1 -- 172.21.15.112:0/2569352171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be395de300 con 0x55be392194d0 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.798+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be391f8d80 0x55be395dddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff13bfff640 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395e1210 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff134011830 tx=0x7ff134013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3652799fe90447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff1217fa640 1 -- 172.21.15.112:0/2569352171 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff134004d20 con 0x55be392194d0 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff1217fa640 1 -- 172.21.15.112:0/2569352171 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff13401edf0 con 0x55be392194d0 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff1217fa640 1 -- 172.21.15.112:0/2569352171 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1340057a0 con 0x55be392194d0 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 -- 172.21.15.112:0/2569352171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 msgr2=0x55be395e1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395e1210 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff134011830 tx=0x7ff134013000 comp rx=0 tx=0).stop 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 -- 172.21.15.112:0/2569352171 shutdown_connections 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be392112f0 0x55be395ea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392194d0 0x55be395e1210 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 --2- 172.21.15.112:0/2569352171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be391f8d80 0x55be395dddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.799+0000 7ff148faa740 1 -- 172.21.15.112:0/2569352171 >> 172.21.15.112:0/2569352171 conn(0x55be3922e300 msgr2=0x55be395dd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 -- 172.21.15.112:0/2569352171 shutdown_connections 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 -- 172.21.15.112:0/2569352171 wait complete. 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 -- start start 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 Processor -- start 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be391f8d80 0x55be395ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395eff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392194d0 0x55be39656820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1ae230 con 0x55be392112f0 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1af6a0 con 0x55be392194d0 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c1ae900 con 0x55be391f8d80 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff148faa740 10 client.0 _ll_register_callbacks cb 0x7ff12c01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.800+0000 7ff13affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:52038/0 (socket says 172.21.15.112:52038) 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 -- 172.21.15.112:0/3054771011 learned_addr learned my addr 172.21.15.112:0/3054771011 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13bfff640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392194d0 0x55be39656820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be391f8d80 0x55be395ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 -- 172.21.15.112:0/3054771011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be391f8d80 msgr2=0x55be395ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be391f8d80 0x55be395ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 -- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392194d0 msgr2=0x55be39656820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392194d0 0x55be39656820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 -- 172.21.15.112:0/3054771011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be39656d60 con 0x55be392112f0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff13affd640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395eff20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff1300c7350 tx=0x7ff130009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f937857575c485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff11affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff11affd640 1 -- 172.21.15.112:0/3054771011 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff130035040 con 0x55be392112f0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff11affd640 1 -- 172.21.15.112:0/3054771011 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8600+0+0 (secure 0 0 0) 0x7ff1300c3650 con 0x55be392112f0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff13401edf0 con 0x55be392112f0 2024-04-25T16:51:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff11affd640 1 -- 172.21.15.112:0/3054771011 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff1300c6d10 con 0x55be392112f0 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff148faa740 10 client.16503 fetch_fsmap learned FSMap version 36 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55be3965e0a0 con 0x55be392112f0 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.801+0000 7ff11affd640 1 -- 172.21.15.112:0/3054771011 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff13401edf0 con 0x55be392112f0 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff11affd640 1 -- 172.21.15.112:0/3054771011 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7ff13004c060 con 0x55be392112f0 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff148faa740 10 client.16503 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff148faa740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff148faa740 20 client.16503 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff148faa740 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be3923a1b0 0x55be39660790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff148faa740 4 client.16503 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be39547ee0 con 0x55be3923a1b0 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be3923a1b0 0x55be39660790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff11affd640 10 client.16503 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:41.802+0000 7ff13b7fe640 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be3923a1b0 0x55be39660790 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b1a8041058683c server_cookie=e8c8aebb7b70359d in_seq=0 out_seq=0 2024-04-25T16:51:41.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1372218 9.92 MB/sec execute 587 sec latency 85.166 ms 2024-04-25T16:51:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff11affd640 1 -- 172.21.15.112:0/3054771011 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be39547ee0 con 0x55be3923a1b0 2024-04-25T16:51:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff11affd640 10 client.16503 handle_command_reply: tid=0 2024-04-25T16:51:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 client.16503 shutdown 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 msgr2=0x55be395eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395eff20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff1300c7350 tx=0x7ff130009620 comp rx=0 tx=0).stop 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 shutdown_connections 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55be392194d0 0x55be39656820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55be392112f0 0x55be395eff20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55be3923a1b0 0x55be39660790 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff11affd640 0 client.16503 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 --2- 172.21.15.112:0/3054771011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55be391f8d80 0x55be395ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 >> 172.21.15.112:0/3054771011 conn(0x55be3922e300 msgr2=0x55be395e5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 shutdown_connections 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.010+0000 7ff148faa740 1 -- 172.21.15.112:0/3054771011 wait complete. 2024-04-25T16:51:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff148faa740 20 client.16503 trim_cache size 0 max 16384 2024-04-25T16:51:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:51:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:51:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:51:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff1240afc50 msgr2=0x7ff1240b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff1240afc50 0x7ff1240b2160 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55be395d6c60 tx=0x7ff134004a40 comp rx=0 tx=0).stop 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 msgr2=0x7ff13c1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 0x7ff13c1a0a50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff130006010 tx=0x7ff130004060 comp rx=0 tx=0).stop 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 shutdown_connections 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff1240afc50 0x7ff1240b2160 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff13c108f00 0x7ff13c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff13c108a60 0x7ff13c1a0a50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 --2- 172.21.15.112:0/552221984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff13c1031c0 0x7ff13c1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.011+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 >> 172.21.15.112:0/552221984 conn(0x7ff13c0781b0 msgr2=0x7ff13c06a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.012+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 shutdown_connections 2024-04-25T16:51:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:42.012+0000 7ff1420f9640 1 -- 172.21.15.112:0/552221984 wait complete. 2024-04-25T16:51:42.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1374269 9.92 MB/sec execute 588 sec latency 91.037 ms 2024-04-25T16:51:43.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:43 smithi116 ceph-mon[32712]: pgmap v422: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-04-25T16:51:43.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:51:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setuser ceph since I am not root 2024-04-25T16:51:43.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:51:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: ignoring --setgroup ceph since I am not root 2024-04-25T16:51:43.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:51:43 smithi116 ceph-mds[62441]: -- 172.21.15.116:0/2219071011 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555adefe1a40 con 0x555adf029600 2024-04-25T16:51:43.508 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:51:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: starting mds.k at 2024-04-25T16:51:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:43 smithi112 ceph-mon[29122]: pgmap v422: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-04-25T16:51:43.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:43 smithi133 ceph-mon[32691]: pgmap v422: 97 pgs: 97 active+clean; 80 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-04-25T16:51:43.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1375033 9.91 MB/sec execute 589 sec latency 32.145 ms 2024-04-25T16:51:44.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:44 smithi116 ceph-mon[32712]: daemon mds.k restarted 2024-04-25T16:51:44.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:44 smithi112 ceph-mon[29122]: daemon mds.k restarted 2024-04-25T16:51:44.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:44 smithi133 ceph-mon[32691]: daemon mds.k restarted 2024-04-25T16:51:44.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1376583 9.93 MB/sec execute 590 sec latency 66.650 ms 2024-04-25T16:51:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:45 smithi116 ceph-mon[32712]: pgmap v423: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-04-25T16:51:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:45 smithi116 ceph-mon[32712]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T16:51:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:45 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6832/1994270531,v1:172.21.15.116:6833/1994270531] up:boot 2024-04-25T16:51:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:45 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:51:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:45 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:51:45.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:45 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:51:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:45 smithi112 ceph-mon[29122]: pgmap v423: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-04-25T16:51:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:45 smithi112 ceph-mon[29122]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T16:51:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:45 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6832/1994270531,v1:172.21.15.116:6833/1994270531] up:boot 2024-04-25T16:51:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:45 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:51:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:45 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:51:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:45 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:51:45.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:45 smithi133 ceph-mon[32691]: pgmap v423: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-04-25T16:51:45.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:45 smithi133 ceph-mon[32691]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T16:51:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:45 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6832/1994270531,v1:172.21.15.116:6833/1994270531] up:boot 2024-04-25T16:51:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:45 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T16:51:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:45 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T16:51:45.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:45 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:51:45.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1378824 9.94 MB/sec execute 591 sec latency 17.924 ms 2024-04-25T16:51:46.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1380367 9.93 MB/sec execute 592 sec latency 71.112 ms 2024-04-25T16:51:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:47 smithi116 ceph-mon[32712]: pgmap v425: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-04-25T16:51:47.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:47 smithi112 ceph-mon[29122]: pgmap v425: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-04-25T16:51:47.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:47 smithi133 ceph-mon[32691]: pgmap v425: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-04-25T16:51:47.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1382460 9.94 MB/sec execute 593 sec latency 97.177 ms 2024-04-25T16:51:48.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:48 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:51:48.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:48 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:51:48.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:48 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:51:48.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1384226 9.93 MB/sec execute 594 sec latency 21.925 ms 2024-04-25T16:51:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:49 smithi116 ceph-mon[32712]: pgmap v426: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-04-25T16:51:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:49 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:51:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:49 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:51:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:49 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:51:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:49 smithi112 ceph-mon[29122]: pgmap v426: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-04-25T16:51:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:49 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:51:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:49 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:51:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:49 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:51:49.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:49 smithi133 ceph-mon[32691]: pgmap v426: 97 pgs: 97 active+clean; 70 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 266 op/s 2024-04-25T16:51:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:49 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:51:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:49 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:51:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:49 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:51:49.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1386216 9.94 MB/sec execute 595 sec latency 15.839 ms 2024-04-25T16:51:50.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1388267 9.93 MB/sec execute 596 sec latency 17.098 ms 2024-04-25T16:51:51.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:51 smithi116 ceph-mon[32712]: pgmap v427: 97 pgs: 97 active+clean; 76 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:51:51.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:51 smithi112 ceph-mon[29122]: pgmap v427: 97 pgs: 97 active+clean; 76 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:51:51.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:51 smithi133 ceph-mon[32691]: pgmap v427: 97 pgs: 97 active+clean; 76 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:51:51.900 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1389925 9.94 MB/sec execute 597 sec latency 22.904 ms 2024-04-25T16:51:52.901 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1391717 9.93 MB/sec execute 598 sec latency 78.555 ms 2024-04-25T16:51:53.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:53 smithi116 ceph-mon[32712]: pgmap v428: 97 pgs: 97 active+clean; 76 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:51:53.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:53 smithi112 ceph-mon[29122]: pgmap v428: 97 pgs: 97 active+clean; 76 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:51:53.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:53 smithi133 ceph-mon[32691]: pgmap v428: 97 pgs: 97 active+clean; 76 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-25T16:51:53.901 INFO:tasks.workunit.client.0.smithi112.stdout: 1 1393678 9.94 MB/sec execute 599 sec latency 14.833 ms 2024-04-25T16:51:54.901 INFO:tasks.workunit.client.0.smithi112.stdout: 1 cleanup 600 sec 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: 0 cleanup 600 sec 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: Operation Count AvgLat MaxLat 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: ---------------------------------------- 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: NTCreateX 189576 1.155 156.511 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: Close 139305 0.003 0.161 2024-04-25T16:51:55.094 INFO:tasks.workunit.client.0.smithi112.stdout: Rename 8029 5.381 181.400 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Unlink 38245 1.623 120.206 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Deltree 6 90.883 156.052 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Mkdir 3 0.008 0.014 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Qpathinfo 171858 1.097 237.829 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Qfileinfo 30172 0.001 0.026 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Qfsinfo 31497 0.226 12.239 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Sfileinfo 15444 0.019 9.342 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Find 66462 0.250 7528.202 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: WriteX 94760 0.057 87.551 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: ReadX 297271 0.021 41.302 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: LockX 618 1.008 5.091 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: UnlockX 618 0.834 4.045 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: Flush 13281 3.538 133.070 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T16:51:55.095 INFO:tasks.workunit.client.0.smithi112.stdout:Throughput 9.93646 MB/sec 1 clients 1 procs max_latency=7528.210 ms 2024-04-25T16:51:55.096 INFO:tasks.workunit.client.0.smithi112.stderr:+ dbench 10 2024-04-25T16:51:55.096 INFO:tasks.workunit.client.0.smithi112.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T16:51:55.096 INFO:tasks.workunit.client.0.smithi112.stdout: 2024-04-25T16:51:55.096 INFO:tasks.workunit.client.0.smithi112.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T16:51:55.096 INFO:tasks.workunit.client.0.smithi112.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.116+0000 7f61163e3640 1 -- 172.21.15.112:0/3608956404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108f20 msgr2=0x7f6110102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.116+0000 7f61163e3640 1 --2- 172.21.15.112:0/3608956404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108f20 0x7f6110102b60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6104009900 tx=0x7f610402a410 comp rx=0 tx=0).stop 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 -- 172.21.15.112:0/3608956404 shutdown_connections 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 --2- 172.21.15.112:0/3608956404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61101031d0 0x7f611010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 --2- 172.21.15.112:0/3608956404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108f20 0x7f6110102b60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 --2- 172.21.15.112:0/3608956404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6110108a80 0x7f6110102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 -- 172.21.15.112:0/3608956404 >> 172.21.15.112:0/3608956404 conn(0x7f6110078150 msgr2=0x7f6110078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 -- 172.21.15.112:0/3608956404 shutdown_connections 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 -- 172.21.15.112:0/3608956404 wait complete. 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 Processor -- start 2024-04-25T16:51:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.117+0000 7f61163e3640 1 -- start start 2024-04-25T16:51:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f61163e3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61101031d0 0x7f611019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f61163e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 0x7f611019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f61163e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6110108f20 0x7f611019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f61163e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6110112c70 con 0x7f6110108a80 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f61163e3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6110112af0 con 0x7f6110108f20 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f61163e3640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6110112df0 con 0x7f61101031d0 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 0x7f611019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 0x7f611019ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45522/0 (socket says 172.21.15.112:45522) 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 -- 172.21.15.112:0/735649187 learned_addr learned my addr 172.21.15.112:0/735649187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f6114980640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6110108f20 0x7f611019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610ffff640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61101031d0 0x7f611019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 -- 172.21.15.112:0/735649187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61101031d0 msgr2=0x7f611019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61101031d0 0x7f611019a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 -- 172.21.15.112:0/735649187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6110108f20 msgr2=0x7f611019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6110108f20 0x7f611019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 -- 172.21.15.112:0/735649187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611019f640 con 0x7f6110108a80 2024-04-25T16:51:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610f7fe640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 0x7f611019ad40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6104006010 tx=0x7f6104004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610d7fa640 1 -- 172.21.15.112:0/735649187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610403f030 con 0x7f6110108a80 2024-04-25T16:51:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.118+0000 7f610d7fa640 1 -- 172.21.15.112:0/735649187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6104003aa0 con 0x7f6110108a80 2024-04-25T16:51:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.119+0000 7f610d7fa640 1 -- 172.21.15.112:0/735649187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104035070 con 0x7f6110108a80 2024-04-25T16:51:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.119+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611019f8d0 con 0x7f6110108a80 2024-04-25T16:51:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.119+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611019fe10 con 0x7f6110108a80 2024-04-25T16:51:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 Processor -- start 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 -- start start 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f610d7fa640 1 -- 172.21.15.112:0/735649187 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f6104042030 con 0x7f6110108a80 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e89030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e89570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558697def460 con 0x558697ae91b0 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558697da9590 con 0x558697ac84d0 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.121+0000 7f611d36a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558697deb420 con 0x558697ac02f0 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e88af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45526/0 (socket says 172.21.15.112:45526) 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 -- 172.21.15.112:0/2194627884 learned_addr learned my addr 172.21.15.112:0/2194627884 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e89570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f610ffff640 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e89030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 -- 172.21.15.112:0/2194627884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 msgr2=0x558697e89030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f610d7fa640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f60f80afcd0 0x7f60f80b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e89030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 -- 172.21.15.112:0/2194627884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 msgr2=0x558697e89570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e89570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 -- 172.21.15.112:0/2194627884 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558697db60d0 con 0x558697ae91b0 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f610d7fa640 1 -- 172.21.15.112:0/735649187 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f61040ff4a0 con 0x7f6110108a80 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f60f80afcd0 0x7f60f80b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f6114980640 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e88af0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6108009fd0 tx=0x7f6108009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7a799edda44545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f60f67fc640 1 -- 172.21.15.112:0/2194627884 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6108003df0 con 0x558697ae91b0 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f60f67fc640 1 -- 172.21.15.112:0/2194627884 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6108004800 con 0x558697ae91b0 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.122+0000 7f60f67fc640 1 -- 172.21.15.112:0/2194627884 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610801b6a0 con 0x558697ae91b0 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 -- 172.21.15.112:0/2194627884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 msgr2=0x558697e88af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e88af0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6108009fd0 tx=0x7f6108009500 comp rx=0 tx=0).stop 2024-04-25T16:51:55.126 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f6114980640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f60f80afcd0 0x7f60f80b21e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6108021000 tx=0x7f610801bd70 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 -- 172.21.15.112:0/2194627884 shutdown_connections 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e89030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e89570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 --2- 172.21.15.112:0/2194627884 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e88af0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 -- 172.21.15.112:0/2194627884 >> 172.21.15.112:0/2194627884 conn(0x558697add300 msgr2=0x558697e852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 -- 172.21.15.112:0/2194627884 shutdown_connections 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.123+0000 7f611d36a740 1 -- 172.21.15.112:0/2194627884 wait complete. 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 -- start start 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 Processor -- start 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e93190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e990a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558697e95c20 con 0x558697ac84d0 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558697d76b60 con 0x558697ac02f0 2024-04-25T16:51:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558697e95da0 con 0x558697ae91b0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f611d36a740 10 client.0 _ll_register_callbacks cb 0x558697d9b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f610ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e93190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f610ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e93190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:33552/0 (socket says 172.21.15.112:33552) 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45542/0 (socket says 172.21.15.112:45542) 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f610ffff640 1 -- 172.21.15.112:0/3934127313 learned_addr learned my addr 172.21.15.112:0/3934127313 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.124+0000 7f6114980640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610ffff640 1 -- 172.21.15.112:0/3934127313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 msgr2=0x558697e990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610ffff640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610ffff640 1 -- 172.21.15.112:0/3934127313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 msgr2=0x558697e936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610ffff640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610ffff640 1 -- 172.21.15.112:0/3934127313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558697b01210 con 0x558697ac02f0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610f7fe640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f610ffff640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e93190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f610000be40 tx=0x7f6100012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320c3db3bb1da8ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f60ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f60ef7fe640 1 -- 172.21.15.112:0/3934127313 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61000035f0 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6108004800 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f60ef7fe640 1 -- 172.21.15.112:0/3934127313 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f6100003790 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f60ef7fe640 1 -- 172.21.15.112:0/3934127313 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6100003990 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f611d36a740 10 client.26204 fetch_fsmap learned FSMap version 38 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558697bcd620 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.125+0000 7f60ef7fe640 1 -- 172.21.15.112:0/3934127313 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6108004800 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f60ef7fe640 1 -- 172.21.15.112:0/3934127313 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 38) ==== 8338+0+0 (secure 0 0 0) 0x7f6100019070 con 0x558697ac02f0 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f611d36a740 10 client.26204 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f611d36a740 10 client.26204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f611d36a740 20 client.26204 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f611d36a740 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697aa7d80 0x558697ea5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f611d36a740 4 client.26204 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558697dfd070 con 0x558697aa7d80 2024-04-25T16:51:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f610f7fe640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697aa7d80 0x558697ea5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f60ef7fe640 10 client.26204 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.126+0000 7f610f7fe640 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697aa7d80 0x558697ea5780 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee798fb123aee2ea server_cookie=a36159e80b2869d7 in_seq=0 out_seq=0 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f60ef7fe640 1 -- 172.21.15.112:0/3934127313 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558697dfd070 con 0x558697aa7d80 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f60ef7fe640 10 client.26204 handle_command_reply: tid=0 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 client.26204 shutdown 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 msgr2=0x558697e93190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e93190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f610000be40 tx=0x7f6100012820 comp rx=0 tx=0).stop 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 shutdown_connections 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697aa7d80 0x558697ea5780 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f60ef7fe640 0 client.26204 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:55.131 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 --2- 172.21.15.112:0/3934127313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e93190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 >> 172.21.15.112:0/3934127313 conn(0x558697add300 msgr2=0x558697e88920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 shutdown_connections 2024-04-25T16:51:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 1 -- 172.21.15.112:0/3934127313 wait complete. 2024-04-25T16:51:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.128+0000 7f611d36a740 20 client.26204 trim_cache size 0 max 16384 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.144+0000 7f611d36a740 1 Processor -- start 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.144+0000 7f611d36a740 1 -- start start 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.144+0000 7f611d36a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e92280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.144+0000 7f611d36a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f611d36a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f611d36a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61101b0260 con 0x558697ae91b0 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f611d36a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61101adb30 con 0x558697ac84d0 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f611d36a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61101ab980 con 0x558697ac02f0 2024-04-25T16:51:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f6114980640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f6114980640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45556/0 (socket says 172.21.15.112:45556) 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f6114980640 1 -- 172.21.15.112:0/3401585730 learned_addr learned my addr 172.21.15.112:0/3401585730 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610ffff640 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e92280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 -- 172.21.15.112:0/3401585730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 msgr2=0x558697e92280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e92280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 -- 172.21.15.112:0/3401585730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 msgr2=0x558697e952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 -- 172.21.15.112:0/3401585730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558697e914d0 con 0x558697ac84d0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f6114980640 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f610f7fe640 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e94db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f61040fcbd0 tx=0x7f6104101a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661fab4239cbf813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f60f5ffb640 1 -- 172.21.15.112:0/3401585730 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104049040 con 0x558697ac84d0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f60f5ffb640 1 -- 172.21.15.112:0/3401585730 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61040c7130 con 0x558697ac84d0 2024-04-25T16:51:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.145+0000 7f60f5ffb640 1 -- 172.21.15.112:0/3401585730 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61040cb9d0 con 0x558697ac84d0 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 -- 172.21.15.112:0/3401585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 msgr2=0x558697e94db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e94db0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f61040fcbd0 tx=0x7f6104101a20 comp rx=0 tx=0).stop 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 -- 172.21.15.112:0/3401585730 shutdown_connections 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ae91b0 0x558697e952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac84d0 0x558697e94db0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 --2- 172.21.15.112:0/3401585730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ac02f0 0x558697e92280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 -- 172.21.15.112:0/3401585730 >> 172.21.15.112:0/3401585730 conn(0x558697add300 msgr2=0x558697e946a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 -- 172.21.15.112:0/3401585730 shutdown_connections 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 -- 172.21.15.112:0/3401585730 wait complete. 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.146+0000 7f611d36a740 1 -- start start 2024-04-25T16:51:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 Processor -- start 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e9f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e9fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e9cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f61101abff0 con 0x558697ac84d0 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61101ad460 con 0x558697ac02f0 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61101ac6c0 con 0x558697ae91b0 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f611d36a740 10 client.0 _ll_register_callbacks cb 0x7f610000b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e9fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610ffff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e9f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e9fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45564/0 (socket says 172.21.15.112:45564) 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 -- 172.21.15.112:0/2055239285 learned_addr learned my addr 172.21.15.112:0/2055239285 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f6114980640 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e9cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 -- 172.21.15.112:0/2055239285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 msgr2=0x558697e9cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e9cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 -- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 msgr2=0x558697e9f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e9f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.147+0000 7f610f7fe640 1 -- 172.21.15.112:0/2055239285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558697e9d410 con 0x558697ac84d0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f610f7fe640 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e9fe70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f61040fcc80 tx=0x7f6104003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88d87fe0583b80a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f60ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f60ef7fe640 1 -- 172.21.15.112:0/2055239285 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6104049050 con 0x558697ac84d0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f60ef7fe640 1 -- 172.21.15.112:0/2055239285 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8556+0+0 (secure 0 0 0) 0x7f61040fce50 con 0x558697ac84d0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61040c7130 con 0x558697ac84d0 2024-04-25T16:51:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f60ef7fe640 1 -- 172.21.15.112:0/2055239285 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f61040c6f70 con 0x558697ac84d0 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 10 client.16527 fetch_fsmap learned FSMap version 38 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558697f0d550 con 0x558697ac84d0 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f60ef7fe640 1 -- 172.21.15.112:0/2055239285 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61040c7130 con 0x558697ac84d0 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f60ef7fe640 1 -- 172.21.15.112:0/2055239285 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 38) ==== 8338+0+0 (secure 0 0 0) 0x7f61040c38b0 con 0x558697ac84d0 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 10 client.16527 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 20 client.16527 populate_metadata read hostname 'smithi112' 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697a7d1c0 0x558697f0fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 4 client.16527 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558697df6ee0 con 0x558697a7d1c0 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.148+0000 7f610ffff640 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697a7d1c0 0x558697f0fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.149+0000 7f60ef7fe640 10 client.16527 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.149+0000 7f610ffff640 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697a7d1c0 0x558697f0fc70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a7a0d9c1c51550 server_cookie=a9173e4745991028 in_seq=0 out_seq=0 2024-04-25T16:51:55.201 INFO:tasks.workunit.client.0.smithi112.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-25T16:51:55.201 INFO:tasks.workunit.client.0.smithi112.stdout:releasing clients 2024-04-25T16:51:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f60ef7fe640 1 -- 172.21.15.112:0/2055239285 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558697df6ee0 con 0x558697a7d1c0 2024-04-25T16:51:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f60ef7fe640 10 client.16527 handle_command_reply: tid=0 2024-04-25T16:51:55.388 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 client.16527 shutdown 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 msgr2=0x558697e9fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e9fe70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f61040fcc80 tx=0x7f6104003f50 comp rx=0 tx=0).stop 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 shutdown_connections 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558697a7d1c0 0x558697f0fc70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558697ae91b0 0x558697e9cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558697ac84d0 0x558697e9fe70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f60ef7fe640 0 client.16527 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 --2- 172.21.15.112:0/2055239285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558697ac02f0 0x558697e9f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 >> 172.21.15.112:0/2055239285 conn(0x558697add300 msgr2=0x558697e91fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 shutdown_connections 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 1 -- 172.21.15.112:0/2055239285 wait complete. 2024-04-25T16:51:55.389 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.390+0000 7f611d36a740 20 client.16527 trim_cache size 0 max 16384 2024-04-25T16:51:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:51:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:51:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:51:55.390 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f60f80afcd0 msgr2=0x7f60f80b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f60f80afcd0 0x7f60f80b21e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6108021000 tx=0x7f610801bd70 comp rx=0 tx=0).stop 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 msgr2=0x7f611019ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 0x7f611019ad40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6104006010 tx=0x7f6104004060 comp rx=0 tx=0).stop 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 shutdown_connections 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f60f80afcd0 0x7f60f80b21e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6110108f20 0x7f611019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6110108a80 0x7f611019ad40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 --2- 172.21.15.112:0/735649187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61101031d0 0x7f611019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.391+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 >> 172.21.15.112:0/735649187 conn(0x7f6110078150 msgr2=0x7f6110106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.392+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 shutdown_connections 2024-04-25T16:51:55.391 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:51:55.392+0000 7f61163e3640 1 -- 172.21.15.112:0/735649187 wait complete. 2024-04-25T16:51:55.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:55 smithi116 ceph-mon[32712]: pgmap v429: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-04-25T16:51:55.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:55 smithi112 ceph-mon[29122]: pgmap v429: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-04-25T16:51:55.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:55 smithi133 ceph-mon[32691]: pgmap v429: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-04-25T16:51:56.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 295 109.13 MB/sec warmup 1 sec latency 183.377 ms 2024-04-25T16:51:57.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 559 95.53 MB/sec warmup 2 sec latency 217.055 ms 2024-04-25T16:51:57.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:57 smithi116 ceph-mon[32712]: pgmap v430: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-25T16:51:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:57 smithi112 ceph-mon[29122]: pgmap v430: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-25T16:51:57.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:57 smithi133 ceph-mon[32691]: pgmap v430: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 164 op/s 2024-04-25T16:51:58.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 915 70.32 MB/sec warmup 3 sec latency 288.414 ms 2024-04-25T16:51:59.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 1370 62.25 MB/sec warmup 4 sec latency 221.909 ms 2024-04-25T16:51:59.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:51:59 smithi116 ceph-mon[32712]: pgmap v431: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-04-25T16:51:59.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:51:59 smithi112 ceph-mon[29122]: pgmap v431: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-04-25T16:51:59.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:51:59 smithi133 ceph-mon[32691]: pgmap v431: 97 pgs: 97 active+clean; 81 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-04-25T16:52:00.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 2132 52.94 MB/sec warmup 5 sec latency 209.333 ms 2024-04-25T16:52:01.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 2441 48.02 MB/sec warmup 6 sec latency 210.464 ms 2024-04-25T16:52:01.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:01 smithi112 ceph-mon[29122]: pgmap v432: 97 pgs: 97 active+clean; 78 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 354 op/s 2024-04-25T16:52:01.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:01 smithi133 ceph-mon[32691]: pgmap v432: 97 pgs: 97 active+clean; 78 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 354 op/s 2024-04-25T16:52:01.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:01 smithi116 ceph-mon[32712]: pgmap v432: 97 pgs: 97 active+clean; 78 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 354 op/s 2024-04-25T16:52:02.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 2956 46.74 MB/sec warmup 7 sec latency 952.166 ms 2024-04-25T16:52:02.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:02 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:52:02.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:02 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:52:02.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:02 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:52:03.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3084 42.55 MB/sec warmup 8 sec latency 1952.209 ms 2024-04-25T16:52:03.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:03 smithi112 ceph-mon[29122]: pgmap v433: 97 pgs: 97 active+clean; 78 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-04-25T16:52:03.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:03 smithi133 ceph-mon[32691]: pgmap v433: 97 pgs: 97 active+clean; 78 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-04-25T16:52:03.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:03 smithi116 ceph-mon[32712]: pgmap v433: 97 pgs: 97 active+clean; 78 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 2.7 MiB/s wr, 274 op/s 2024-04-25T16:52:04.201 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3507 41.54 MB/sec warmup 9 sec latency 2952.237 ms 2024-04-25T16:52:05.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3680 37.85 MB/sec warmup 10 sec latency 3952.265 ms 2024-04-25T16:52:05.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:05 smithi112 ceph-mon[29122]: pgmap v434: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-04-25T16:52:05.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:05 smithi133 ceph-mon[32691]: pgmap v434: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-04-25T16:52:05.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:05 smithi116 ceph-mon[32712]: pgmap v434: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 513 op/s 2024-04-25T16:52:06.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 3899 34.93 MB/sec warmup 11 sec latency 4952.334 ms 2024-04-25T16:52:07.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4027 32.27 MB/sec warmup 12 sec latency 5952.382 ms 2024-04-25T16:52:07.560 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:07 smithi116 ceph-mon[32712]: pgmap v435: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.7 MiB/s wr, 437 op/s 2024-04-25T16:52:07.560 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:07 smithi116 ceph-mon[32712]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T16:52:07.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:07 smithi112 ceph-mon[29122]: pgmap v435: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.7 MiB/s wr, 437 op/s 2024-04-25T16:52:07.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:07 smithi112 ceph-mon[29122]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T16:52:07.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:07 smithi133 ceph-mon[32691]: pgmap v435: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 3.7 MiB/s wr, 437 op/s 2024-04-25T16:52:07.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:07 smithi133 ceph-mon[32691]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T16:52:08.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4058 29.86 MB/sec warmup 13 sec latency 6952.421 ms 2024-04-25T16:52:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 -- 172.21.15.112:0/2763756916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b011a7c0 msgr2=0x7f85b011cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/2763756916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b011a7c0 0x7f85b011cbe0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009ca0 tx=0x7f85a802a960 comp rx=0 tx=0).stop 2024-04-25T16:52:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 -- 172.21.15.112:0/2763756916 shutdown_connections 2024-04-25T16:52:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/2763756916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b011d120 0x7f85b011f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/2763756916 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b011a7c0 0x7f85b011cbe0 secure :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009ca0 tx=0x7f85a802a960 comp rx=0 tx=0).stop 2024-04-25T16:52:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/2763756916 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85b010a0d0 0x7f85b010a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.548+0000 7f85b4f0d640 1 -- 172.21.15.112:0/2763756916 >> 172.21.15.112:0/2763756916 conn(0x7f85b006e7d0 msgr2=0x7f85b006ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.551+0000 7f85b4f0d640 1 -- 172.21.15.112:0/2763756916 shutdown_connections 2024-04-25T16:52:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.551+0000 7f85b4f0d640 1 -- 172.21.15.112:0/2763756916 wait complete. 2024-04-25T16:52:08.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.552+0000 7f85b4f0d640 1 Processor -- start 2024-04-25T16:52:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.553+0000 7f85b4f0d640 1 -- start start 2024-04-25T16:52:08.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85b4f0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 0x7f85b01a9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85b4f0d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85b011d120 0x7f85b01a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85b4f0d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01adfd0 0x7f85b01ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85b4f0d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0122c10 con 0x7f85b010a0d0 2024-04-25T16:52:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85b4f0d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0122a90 con 0x7f85b011d120 2024-04-25T16:52:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85b4f0d640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0122d90 con 0x7f85b01adfd0 2024-04-25T16:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 0x7f85b01a9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85ae59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 0x7f85b01a9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36282/0 (socket says 172.21.15.112:36282) 2024-04-25T16:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.554+0000 7f85ae59c640 1 -- 172.21.15.112:0/1013012585 learned_addr learned my addr 172.21.15.112:0/1013012585 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.555+0000 7f85ae59c640 1 -- 172.21.15.112:0/1013012585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01adfd0 msgr2=0x7f85b01ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.555+0000 7f85ae59c640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01adfd0 0x7f85b01ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.555+0000 7f85ae59c640 1 -- 172.21.15.112:0/1013012585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85b011d120 msgr2=0x7f85b01a9dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T16:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.555+0000 7f85add9b640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85b011d120 0x7f85b01a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.555+0000 7f85ae59c640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85b011d120 0x7f85b01a9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.555+0000 7f85ae59c640 1 -- 172.21.15.112:0/1013012585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b01aeae0 con 0x7f85b010a0d0 2024-04-25T16:52:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.556+0000 7f85ae59c640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 0x7f85b01a9870 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f85a400b4a0 tx=0x7f85a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.556 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.559+0000 7f859f7fe640 1 -- 172.21.15.112:0/1013012585 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4014070 con 0x7f85b010a0d0 2024-04-25T16:52:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.559+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b01aedd0 con 0x7f85b010a0d0 2024-04-25T16:52:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.559+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b0112330 con 0x7f85b010a0d0 2024-04-25T16:52:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.559+0000 7f859f7fe640 1 -- 172.21.15.112:0/1013012585 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a4010ac0 con 0x7f85b010a0d0 2024-04-25T16:52:08.557 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.559+0000 7f859f7fe640 1 -- 172.21.15.112:0/1013012585 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a4017c30 con 0x7f85b010a0d0 2024-04-25T16:52:08.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.563+0000 7f859f7fe640 1 -- 172.21.15.112:0/1013012585 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f85a4017070 con 0x7f85b010a0d0 2024-04-25T16:52:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.565+0000 7f859f7fe640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f85980afe60 0x7f85980b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.565+0000 7f859f7fe640 1 -- 172.21.15.112:0/1013012585 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8644+0+0 (secure 0 0 0) 0x7f85a40de260 con 0x7f85b010a0d0 2024-04-25T16:52:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.565+0000 7f85bbdc4740 1 Processor -- start 2024-04-25T16:52:08.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.566+0000 7f85bbdc4740 1 -- start start 2024-04-25T16:52:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.566+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x5563592fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.566+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.566+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563592fe570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.566+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x556359264460 con 0x556358f5e1b0 2024-04-25T16:52:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55635921e590 con 0x556358f3d4d0 2024-04-25T16:52:08.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556359260420 con 0x556358f352f0 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85aed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563592fe570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85aed9d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563592fe570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:39682/0 (socket says 172.21.15.112:39682) 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85aed9d640 1 -- 172.21.15.112:0/945450412 learned_addr learned my addr 172.21.15.112:0/945450412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85add9b640 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85add9b640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f85980afe60 0x7f85980b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.567+0000 7f85ae59c640 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x5563592fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.568+0000 7f85aed9d640 1 -- 172.21.15.112:0/945450412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 msgr2=0x5563592fe030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.566 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.568+0000 7f85aed9d640 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fe030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.568+0000 7f85aed9d640 1 -- 172.21.15.112:0/945450412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 msgr2=0x5563592fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.568+0000 7f85aed9d640 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x5563592fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.568+0000 7f85aed9d640 1 -- 172.21.15.112:0/945450412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55635922b0d0 con 0x556358f3d4d0 2024-04-25T16:52:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.569+0000 7f85add9b640 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:52:08.567 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.569+0000 7f85aed9d640 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563592fe570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009760 tx=0x7f85a0009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2478c5e058560d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.569+0000 7f859cff9640 1 -- 172.21.15.112:0/945450412 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0015020 con 0x556358f3d4d0 2024-04-25T16:52:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.569+0000 7f859cff9640 1 -- 172.21.15.112:0/945450412 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a000bd70 con 0x556358f3d4d0 2024-04-25T16:52:08.568 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.569+0000 7f859cff9640 1 -- 172.21.15.112:0/945450412 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0010380 con 0x556358f3d4d0 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 -- 172.21.15.112:0/945450412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 msgr2=0x5563592fe570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563592fe570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009760 tx=0x7f85a0009b60 comp rx=0 tx=0).stop 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 -- 172.21.15.112:0/945450412 shutdown_connections 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fe030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563592fe570 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/945450412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x5563592fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 -- 172.21.15.112:0/945450412 >> 172.21.15.112:0/945450412 conn(0x556358f52300 msgr2=0x5563592fa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85add9b640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f85980afe60 0x7f85980b2370 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85b011c730 tx=0x7f85a8005fd0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.569 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 -- 172.21.15.112:0/945450412 shutdown_connections 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.571+0000 7f85bbdc4740 1 -- 172.21.15.112:0/945450412 wait complete. 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 -- start start 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 Processor -- start 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563593080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359314120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.570 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55635930ac70 con 0x556358f5e1b0 2024-04-25T16:52:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5563591ebb60 con 0x556358f3d4d0 2024-04-25T16:52:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.572+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55635930adf0 con 0x556358f352f0 2024-04-25T16:52:08.571 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.573+0000 7f85bbdc4740 10 client.0 _ll_register_callbacks cb 0x5563592102b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359314120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359314120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36288/0 (socket says 172.21.15.112:36288) 2024-04-25T16:52:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 -- 172.21.15.112:0/1433394366 learned_addr learned my addr 172.21.15.112:0/1433394366 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 -- 172.21.15.112:0/1433394366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 msgr2=0x5563593080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.573 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563593080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 -- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 msgr2=0x5563593085e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 -- 172.21.15.112:0/1433394366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556358f76210 con 0x556358f5e1b0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.574+0000 7f85aed9d640 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359314120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f85a0011650 tx=0x7f85a0011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4577feee7c8a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.575+0000 7f8595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.575+0000 7f8595ffb640 1 -- 172.21.15.112:0/1433394366 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000b6c0 con 0x556358f5e1b0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.575+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a000bd70 con 0x556358f5e1b0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.575+0000 7f8595ffb640 1 -- 172.21.15.112:0/1433394366 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8644+0+0 (secure 0 0 0) 0x7f85a000b860 con 0x556358f5e1b0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.575+0000 7f8595ffb640 1 -- 172.21.15.112:0/1433394366 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85a000ba60 con 0x556358f5e1b0 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 10 client.16539 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:08.574 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556359042620 con 0x556358f5e1b0 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f8595ffb640 1 -- 172.21.15.112:0/1433394366 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a000bd70 con 0x556358f5e1b0 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f8595ffb640 1 -- 172.21.15.112:0/1433394366 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f85a002b430 con 0x556358f5e1b0 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 10 client.16539 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 20 client.16539 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358f1cd80 0x55635931ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 4 client.16539 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:52:08.575 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.576+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556359272070 con 0x556358f1cd80 2024-04-25T16:52:08.579 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.581+0000 7f85ae59c640 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358f1cd80 0x55635931ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.585+0000 7f85ae59c640 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358f1cd80 0x55635931ab10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372c175d82b4556b server_cookie=19ef43dc6ee3cf7e in_seq=0 out_seq=0 2024-04-25T16:52:08.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.585+0000 7f8595ffb640 10 client.16539 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.589+0000 7f8595ffb640 1 -- 172.21.15.112:0/1433394366 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556359272070 con 0x556358f1cd80 2024-04-25T16:52:08.586 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.589+0000 7f8595ffb640 10 client.16539 handle_command_reply: tid=0 2024-04-25T16:52:08.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.589+0000 7f85bbdc4740 1 client.16539 shutdown 2024-04-25T16:52:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.591+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 msgr2=0x556359314120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.591+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359314120 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f85a0011650 tx=0x7f85a0011680 comp rx=0 tx=0).stop 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 shutdown_connections 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359314120 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358f1cd80 0x55635931ab10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/1433394366 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563593080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 >> 172.21.15.112:0/1433394366 conn(0x556358f52300 msgr2=0x5563592ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f8595ffb640 0 client.16539 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 shutdown_connections 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 1 -- 172.21.15.112:0/1433394366 wait complete. 2024-04-25T16:52:08.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.593+0000 7f85bbdc4740 20 client.16539 trim_cache size 0 max 16384 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.627+0000 7f85bbdc4740 1 Processor -- start 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.627+0000 7f85bbdc4740 1 -- start start 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.627+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.627+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593083e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359313d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f85900112f0 con 0x556358f5e1b0 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f859000ebc0 con 0x556358f3d4d0 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f859000ca30 con 0x556358f352f0 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359313d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85aed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359313d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36302/0 (socket says 172.21.15.112:36302) 2024-04-25T16:52:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85aed9d640 1 -- 172.21.15.112:0/3193650533 learned_addr learned my addr 172.21.15.112:0/3193650533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85add9b640 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593083e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.628+0000 7f85ae59c640 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.629+0000 7f85add9b640 1 -- 172.21.15.112:0/3193650533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 msgr2=0x5563592fa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.629+0000 7f85add9b640 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.629+0000 7f85add9b640 1 -- 172.21.15.112:0/3193650533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 msgr2=0x556359313d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.629+0000 7f85add9b640 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359313d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.629+0000 7f85add9b640 1 -- 172.21.15.112:0/3193650533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556359307190 con 0x556358f3d4d0 2024-04-25T16:52:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.631+0000 7f85add9b640 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593083e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85a8008000 tx=0x7f85a80095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff404e3eb0a4b32c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.632+0000 7f8597fff640 1 -- 172.21.15.112:0/3193650533 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8042070 con 0x556358f3d4d0 2024-04-25T16:52:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.632+0000 7f8597fff640 1 -- 172.21.15.112:0/3193650533 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a8043040 con 0x556358f3d4d0 2024-04-25T16:52:08.629 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.632+0000 7f8597fff640 1 -- 172.21.15.112:0/3193650533 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8034040 con 0x556358f3d4d0 2024-04-25T16:52:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.633+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3193650533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 msgr2=0x5563593083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:08.632 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.633+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593083e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f85a8008000 tx=0x7f85a80095c0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.635+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3193650533 shutdown_connections 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.635+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x556359313d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.635+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f3d4d0 0x5563593083e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.635+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3193650533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f352f0 0x5563592fa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.635+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3193650533 >> 172.21.15.112:0/3193650533 conn(0x556358f52300 msgr2=0x5563592f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.636+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3193650533 shutdown_connections 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.636+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3193650533 wait complete. 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.636+0000 7f85bbdc4740 1 -- start start 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.636+0000 7f85bbdc4740 1 Processor -- start 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f352f0 0x556359310480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 0x55635930b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x55635930b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f859000d080 con 0x556358f5e1b0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f859000e4f0 con 0x556358f352f0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f859000d750 con 0x556358f3d4d0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85bbdc4740 10 client.0 _ll_register_callbacks cb 0x7f85a00095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 0x55635930b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85add9b640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 0x55635930b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:56528/0 (socket says 172.21.15.112:56528) 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85add9b640 1 -- 172.21.15.112:0/3027418029 learned_addr learned my addr 172.21.15.112:0/3027418029 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85aed9d640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x55635930b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.637+0000 7f85ae59c640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f352f0 0x556359310480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85add9b640 1 -- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f352f0 msgr2=0x556359310480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85add9b640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f352f0 0x556359310480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85add9b640 1 -- 172.21.15.112:0/3027418029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 msgr2=0x55635930b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85add9b640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x55635930b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85add9b640 1 -- 172.21.15.112:0/3027418029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55635930bb40 con 0x556358f3d4d0 2024-04-25T16:52:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85aed9d640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x55635930b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f85add9b640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 0x55635930b040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f85a80308c0 tx=0x7f85a80308f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd35d0ab3b58e359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.638+0000 7f8595ffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:52:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.639+0000 7f8595ffb640 1 -- 172.21.15.112:0/3027418029 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8056060 con 0x556358f3d4d0 2024-04-25T16:52:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.639+0000 7f8595ffb640 1 -- 172.21.15.112:0/3027418029 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8644+0+0 (secure 0 0 0) 0x7f85a8030ca0 con 0x556358f3d4d0 2024-04-25T16:52:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.639+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a8043040 con 0x556358f3d4d0 2024-04-25T16:52:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.639+0000 7f8595ffb640 1 -- 172.21.15.112:0/3027418029 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f85a8034050 con 0x556358f3d4d0 2024-04-25T16:52:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.639+0000 7f85bbdc4740 10 client.26131 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.639+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556359381b90 con 0x556358f3d4d0 2024-04-25T16:52:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.641+0000 7f8595ffb640 1 -- 172.21.15.112:0/3027418029 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85a8043040 con 0x556358f3d4d0 2024-04-25T16:52:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.641+0000 7f8595ffb640 1 -- 172.21.15.112:0/3027418029 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f85a8051620 con 0x556358f3d4d0 2024-04-25T16:52:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.642+0000 7f85bbdc4740 10 client.26131 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.642+0000 7f85bbdc4740 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.642+0000 7f85bbdc4740 20 client.26131 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:08.640 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.642+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358ef21c0 0x556359384210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.643+0000 7f85bbdc4740 4 client.26131 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:52:08.641 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.643+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55635926bee0 con 0x556358ef21c0 2024-04-25T16:52:08.643 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.645+0000 7f85ae59c640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358ef21c0 0x556359384210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.652+0000 7f85ae59c640 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358ef21c0 0x556359384210 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2510a2683a166ba server_cookie=f3fcac4bd3387e60 in_seq=0 out_seq=0 2024-04-25T16:52:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:08.652+0000 7f8595ffb640 10 client.26131 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:09.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4059 27.73 MB/sec warmup 14 sec latency 7952.462 ms 2024-04-25T16:52:09.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:09 smithi112 ceph-mon[29122]: pgmap v437: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-04-25T16:52:09.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:09 smithi133 ceph-mon[32691]: pgmap v437: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-04-25T16:52:09.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:09 smithi116 ceph-mon[32712]: pgmap v437: 97 pgs: 97 active+clean; 95 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 525 op/s 2024-04-25T16:52:10.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4084 26.09 MB/sec warmup 15 sec latency 8952.506 ms 2024-04-25T16:52:11.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4422 25.47 MB/sec warmup 16 sec latency 9952.546 ms 2024-04-25T16:52:11.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:11 smithi112 ceph-mon[29122]: pgmap v438: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 276 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 450 op/s 2024-04-25T16:52:11.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:11 smithi133 ceph-mon[32691]: pgmap v438: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 276 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 450 op/s 2024-04-25T16:52:11.777 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:11 smithi116 ceph-mon[32712]: pgmap v438: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 276 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 450 op/s 2024-04-25T16:52:12.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 4939 25.38 MB/sec warmup 17 sec latency 10952.596 ms 2024-04-25T16:52:13.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 5524 25.25 MB/sec warmup 18 sec latency 11952.636 ms 2024-04-25T16:52:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:13 smithi112 ceph-mon[29122]: pgmap v439: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 276 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 450 op/s 2024-04-25T16:52:13.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:13 smithi133 ceph-mon[32691]: pgmap v439: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 276 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 450 op/s 2024-04-25T16:52:13.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:13 smithi116 ceph-mon[32712]: pgmap v439: 97 pgs: 35 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 45 active+clean; 276 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 21 MiB/s wr, 450 op/s 2024-04-25T16:52:14.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6012 26.03 MB/sec warmup 19 sec latency 12952.756 ms 2024-04-25T16:52:15.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6407 26.57 MB/sec warmup 20 sec latency 13952.797 ms 2024-04-25T16:52:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:15 smithi112 ceph-mon[29122]: pgmap v440: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.659+0000 7f8595ffb640 1 -- 172.21.15.112:0/3027418029 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55635926bee0 con 0x556358ef21c0 2024-04-25T16:52:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.659+0000 7f8595ffb640 10 client.26131 handle_command_reply: tid=0 2024-04-25T16:52:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 client.26131 shutdown 2024-04-25T16:52:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 msgr2=0x55635930b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 0x55635930b040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f85a80308c0 tx=0x7f85a80308f0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.658 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 shutdown_connections 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x556358ef21c0 0x556359384210 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556358f3d4d0 0x55635930b040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x556358f5e1b0 0x55635930b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 --2- 172.21.15.112:0/3027418029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556358f352f0 0x556359310480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 >> 172.21.15.112:0/3027418029 conn(0x556358f52300 msgr2=0x5563592ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f8595ffb640 0 client.26131 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 shutdown_connections 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 1 -- 172.21.15.112:0/3027418029 wait complete. 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.660+0000 7f85bbdc4740 20 client.26131 trim_cache size 0 max 16384 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:52:15.659 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:52:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:52:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:52:15.660 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f85980afe60 msgr2=0x7f85980b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f85980afe60 0x7f85980b2370 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85b011c730 tx=0x7f85a8005fd0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 msgr2=0x7f85b01a9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 0x7f85b01a9870 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f85a400b4a0 tx=0x7f85a400b960 comp rx=0 tx=0).stop 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 shutdown_connections 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f85980afe60 0x7f85980b2370 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85b01adfd0 0x7f85b01ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85b011d120 0x7f85b01a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 --2- 172.21.15.112:0/1013012585 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85b010a0d0 0x7f85b01a9870 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 >> 172.21.15.112:0/1013012585 conn(0x7f85b006e7d0 msgr2=0x7f85b011abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 shutdown_connections 2024-04-25T16:52:15.661 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:15.661+0000 7f85b4f0d640 1 -- 172.21.15.112:0/1013012585 wait complete. 2024-04-25T16:52:15.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:15 smithi133 ceph-mon[32691]: pgmap v440: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:15.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:15 smithi116 ceph-mon[32712]: pgmap v440: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:16.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6574 25.93 MB/sec warmup 21 sec latency 14415.901 ms 2024-04-25T16:52:17.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6733 24.92 MB/sec warmup 22 sec latency 216.374 ms 2024-04-25T16:52:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:17 smithi112 ceph-mon[29122]: pgmap v441: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:17.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:17 smithi112 ceph-mon[29122]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T16:52:17.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:17 smithi133 ceph-mon[32691]: pgmap v441: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:17 smithi133 ceph-mon[32691]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T16:52:17.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:17 smithi116 ceph-mon[32712]: pgmap v441: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:17.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:17 smithi116 ceph-mon[32712]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T16:52:18.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 6912 24.04 MB/sec warmup 23 sec latency 129.606 ms 2024-04-25T16:52:19.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 7146 23.27 MB/sec warmup 24 sec latency 101.678 ms 2024-04-25T16:52:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:19 smithi112 ceph-mon[29122]: pgmap v443: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:19.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:19 smithi133 ceph-mon[32691]: pgmap v443: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:19.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:19 smithi116 ceph-mon[32712]: pgmap v443: 97 pgs: 97 active+clean; 233 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 20 MiB/s wr, 550 op/s 2024-04-25T16:52:20.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 7477 22.95 MB/sec warmup 25 sec latency 155.664 ms 2024-04-25T16:52:21.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 7680 22.82 MB/sec warmup 26 sec latency 609.276 ms 2024-04-25T16:52:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:21 smithi112 ceph-mon[29122]: pgmap v444: 97 pgs: 97 active+clean; 221 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-04-25T16:52:21.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:21 smithi133 ceph-mon[32691]: pgmap v444: 97 pgs: 97 active+clean; 221 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-04-25T16:52:21.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:21 smithi116 ceph-mon[32712]: pgmap v444: 97 pgs: 97 active+clean; 221 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-04-25T16:52:22.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 8073 22.30 MB/sec warmup 27 sec latency 188.985 ms 2024-04-25T16:52:23.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 8626 22.06 MB/sec warmup 28 sec latency 226.807 ms 2024-04-25T16:52:23.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:23 smithi112 ceph-mon[29122]: pgmap v445: 97 pgs: 97 active+clean; 221 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-04-25T16:52:23.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:23 smithi133 ceph-mon[32691]: pgmap v445: 97 pgs: 97 active+clean; 221 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-04-25T16:52:23.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:23 smithi116 ceph-mon[32712]: pgmap v445: 97 pgs: 97 active+clean; 221 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 509 op/s 2024-04-25T16:52:24.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 9110 22.85 MB/sec warmup 29 sec latency 190.406 ms 2024-04-25T16:52:25.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 9663 23.38 MB/sec warmup 30 sec latency 163.352 ms 2024-04-25T16:52:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:25 smithi112 ceph-mon[29122]: pgmap v446: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 271 op/s 2024-04-25T16:52:25.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:25 smithi133 ceph-mon[32691]: pgmap v446: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 271 op/s 2024-04-25T16:52:25.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:25 smithi116 ceph-mon[32712]: pgmap v446: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 271 op/s 2024-04-25T16:52:26.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 9920 23.23 MB/sec warmup 31 sec latency 123.811 ms 2024-04-25T16:52:27.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 10157 22.91 MB/sec warmup 32 sec latency 80.430 ms 2024-04-25T16:52:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:27 smithi112 ceph-mon[29122]: pgmap v447: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 271 op/s 2024-04-25T16:52:27.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:27 smithi133 ceph-mon[32691]: pgmap v447: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 271 op/s 2024-04-25T16:52:27.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:27 smithi116 ceph-mon[32712]: pgmap v447: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 271 op/s 2024-04-25T16:52:28.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 10358 22.47 MB/sec warmup 33 sec latency 202.949 ms 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 -- 172.21.15.112:0/3950038191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108a60 msgr2=0x7f0b04102610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3950038191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108a60 0x7f0b04102610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009900 tx=0x7f0b0002a410 comp rx=0 tx=0).stop 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 -- 172.21.15.112:0/3950038191 shutdown_connections 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3950038191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b04108f00 0x7f0b04102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3950038191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b041031c0 0x7f0b0410f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3950038191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108a60 0x7f0b04102610 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 -- 172.21.15.112:0/3950038191 >> 172.21.15.112:0/3950038191 conn(0x7f0b040781b0 msgr2=0x7f0b040785f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 -- 172.21.15.112:0/3950038191 shutdown_connections 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 -- 172.21.15.112:0/3950038191 wait complete. 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 Processor -- start 2024-04-25T16:52:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.763+0000 7f0b0c357640 1 -- start start 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0c357640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b041031c0 0x7f0b0419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0c357640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b04108a60 0x7f0b0419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0c357640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 0x7f0b0419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0c357640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04112c60 con 0x7f0b04108f00 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0c357640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04112ae0 con 0x7f0b04108a60 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0c357640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04112de0 con 0x7f0b041031c0 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 0x7f0b0419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 0x7f0b0419ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46042/0 (socket says 172.21.15.112:46042) 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 -- 172.21.15.112:0/3659320616 learned_addr learned my addr 172.21.15.112:0/3659320616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b04108a60 0x7f0b0419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b041031c0 0x7f0b0419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 -- 172.21.15.112:0/3659320616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b041031c0 msgr2=0x7f0b0419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b041031c0 0x7f0b0419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 -- 172.21.15.112:0/3659320616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b04108a60 msgr2=0x7f0b0419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b04108a60 0x7f0b0419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 -- 172.21.15.112:0/3659320616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0419f590 con 0x7f0b04108f00 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.764+0000 7f0b0a8f4640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 0x7f0b0419ef50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00eb10 tx=0x7f0afc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.765+0000 7f0afb7fe640 1 -- 172.21.15.112:0/3659320616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc004b20 con 0x7f0b04108f00 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.765+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0419f880 con 0x7f0b04108f00 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.765+0000 7f0afb7fe640 1 -- 172.21.15.112:0/3659320616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0afc004cc0 con 0x7f0b04108f00 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.765+0000 7f0afb7fe640 1 -- 172.21.15.112:0/3659320616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc010670 con 0x7f0b04108f00 2024-04-25T16:52:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.765+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0419fd80 con 0x7f0b04108f00 2024-04-25T16:52:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0afb7fe640 1 -- 172.21.15.112:0/3659320616 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f0afc005020 con 0x7f0b04108f00 2024-04-25T16:52:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 Processor -- start 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 -- start start 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 0x5647561c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561c7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56475612d460 con 0x564755e271b0 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5647560e7590 con 0x564755e064d0 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.768+0000 7f0b1320a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564756129420 con 0x564755dfe2f0 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 0x5647561c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 0x5647561c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46046/0 (socket says 172.21.15.112:46046) 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 -- 172.21.15.112:0/3718842695 learned_addr learned my addr 172.21.15.112:0/3718842695 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b0a0f3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0afb7fe640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f0af00afc50 0x7f0af00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 -- 172.21.15.112:0/3718842695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 msgr2=0x5647561c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 -- 172.21.15.112:0/3718842695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 msgr2=0x5647561c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 -- 172.21.15.112:0/3718842695 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647560f40d0 con 0x564755e271b0 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b0a8f4640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f0af00afc50 0x7f0af00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 0x5647561c6af0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0af40135a0 tx=0x7f0af4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9849a9913712e050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0afb7fe640 1 -- 172.21.15.112:0/3659320616 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7f0afc0df670 con 0x7f0b04108f00 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.769+0000 7f0af8bfe640 1 -- 172.21.15.112:0/3718842695 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4005020 con 0x564755e271b0 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.770+0000 7f0af8bfe640 1 -- 172.21.15.112:0/3718842695 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af40047a0 con 0x564755e271b0 2024-04-25T16:52:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.770+0000 7f0af8bfe640 1 -- 172.21.15.112:0/3718842695 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af400f760 con 0x564755e271b0 2024-04-25T16:52:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 -- 172.21.15.112:0/3718842695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 msgr2=0x5647561c6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 0x5647561c6af0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0af40135a0 tx=0x7f0af4005850 comp rx=0 tx=0).stop 2024-04-25T16:52:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b0a8f4640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f0af00afc50 0x7f0af00b2160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0dd370 tx=0x7f0afc0ab040 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 -- 172.21.15.112:0/3718842695 shutdown_connections 2024-04-25T16:52:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3718842695 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e271b0 0x5647561c6af0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 -- 172.21.15.112:0/3718842695 >> 172.21.15.112:0/3718842695 conn(0x564755e1b300 msgr2=0x5647561c32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 -- 172.21.15.112:0/3718842695 shutdown_connections 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.771+0000 7f0b1320a740 1 -- 172.21.15.112:0/3718842695 wait complete. 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 -- start start 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 Processor -- start 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755e271b0 0x5647561d7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5647561d3c20 con 0x564755e064d0 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5647560b4b60 con 0x564755dfe2f0 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5647561d3da0 con 0x564755e271b0 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b1320a740 10 client.0 _ll_register_callbacks cb 0x5647560d92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46052/0 (socket says 172.21.15.112:46052) 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/1558427581 learned_addr learned my addr 172.21.15.112:0/1558427581 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755e271b0 0x5647561d7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/1558427581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755e271b0 msgr2=0x5647561d7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755e271b0 0x5647561d7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 msgr2=0x5647561d11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.772+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561d11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/1558427581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564755e3f210 con 0x564755e064d0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561d16e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009900 tx=0x7f0b00032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfff3d713c3bfb81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0aedffb640 1 -- 172.21.15.112:0/1558427581 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00035040 con 0x564755e064d0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0aedffb640 1 -- 172.21.15.112:0/1558427581 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7f0b00031760 con 0x564755e064d0 2024-04-25T16:52:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af40047a0 con 0x564755e064d0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0aedffb640 1 -- 172.21.15.112:0/1558427581 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b00004040 con 0x564755e064d0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0b1320a740 10 client.16557 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564755f0b620 con 0x564755e064d0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.773+0000 7f0aedffb640 1 -- 172.21.15.112:0/1558427581 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af40047a0 con 0x564755e064d0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0aedffb640 1 -- 172.21.15.112:0/1558427581 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0b00004750 con 0x564755e064d0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b1320a740 10 client.16557 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b1320a740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b1320a740 20 client.16557 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755de5d80 0x5647561e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b1320a740 4 client.16557 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56475613b070 con 0x564755de5d80 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755de5d80 0x5647561e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0aedffb640 10 client.16557 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.774+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755de5d80 0x5647561e3790 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa23d91786b13c8 server_cookie=a91443ef3ea1aab5 in_seq=0 out_seq=0 2024-04-25T16:52:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0aedffb640 1 -- 172.21.15.112:0/1558427581 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56475613b070 con 0x564755de5d80 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0aedffb640 10 client.16557 handle_command_reply: tid=0 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 client.16557 shutdown 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 msgr2=0x5647561d16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561d16e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009900 tx=0x7f0b00032e90 comp rx=0 tx=0).stop 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 shutdown_connections 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755de5d80 0x5647561e3790 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755e271b0 0x5647561d7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561d16e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0aedffb640 0 client.16557 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1558427581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561d11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 >> 172.21.15.112:0/1558427581 conn(0x564755e1b300 msgr2=0x5647561c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 shutdown_connections 2024-04-25T16:52:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 1 -- 172.21.15.112:0/1558427581 wait complete. 2024-04-25T16:52:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.776+0000 7f0b1320a740 20 client.16557 trim_cache size 0 max 16384 2024-04-25T16:52:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.791+0000 7f0b1320a740 1 Processor -- start 2024-04-25T16:52:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.791+0000 7f0b1320a740 1 -- start start 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755de5d80 0x5647561e8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b1320a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041b23c0 con 0x564755e064d0 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b1320a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041afc90 con 0x564755dfe2f0 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b1320a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041adb00 con 0x564755de5d80 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46058/0 (socket says 172.21.15.112:46058) 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 -- 172.21.15.112:0/1081327482 learned_addr learned my addr 172.21.15.112:0/1081327482 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755de5d80 0x5647561e8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 -- 172.21.15.112:0/1081327482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755de5d80 msgr2=0x5647561e8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755de5d80 0x5647561e8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 -- 172.21.15.112:0/1081327482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 msgr2=0x5647561e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 -- 172.21.15.112:0/1081327482 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647561c3e90 con 0x564755e064d0 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.792+0000 7f0b098f2640 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561ca7b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0af40135a0 tx=0x7f0af4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5b056e0619aa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.793+0000 7f0aeffff640 1 -- 172.21.15.112:0/1081327482 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4028040 con 0x564755e064d0 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.793+0000 7f0aeffff640 1 -- 172.21.15.112:0/1081327482 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af4004c00 con 0x564755e064d0 2024-04-25T16:52:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.793+0000 7f0aeffff640 1 -- 172.21.15.112:0/1081327482 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4005650 con 0x564755e064d0 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 -- 172.21.15.112:0/1081327482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 msgr2=0x5647561ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561ca7b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f0af40135a0 tx=0x7f0af4009660 comp rx=0 tx=0).stop 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 -- 172.21.15.112:0/1081327482 shutdown_connections 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755e064d0 0x5647561ca7b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755dfe2f0 0x5647561e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 --2- 172.21.15.112:0/1081327482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755de5d80 0x5647561e8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 -- 172.21.15.112:0/1081327482 >> 172.21.15.112:0/1081327482 conn(0x564755e1b300 msgr2=0x5647561cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 -- 172.21.15.112:0/1081327482 shutdown_connections 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 -- 172.21.15.112:0/1081327482 wait complete. 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.794+0000 7f0b1320a740 1 -- start start 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 Processor -- start 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 0x5647561dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561da830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041ae150 con 0x564755de5d80 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041af5c0 con 0x564755e064d0 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b041ae820 con 0x564755dfe2f0 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b1320a740 10 client.0 _ll_register_callbacks cb 0x7f0b00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 0x5647561dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b098f2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 0x5647561dd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46066/0 (socket says 172.21.15.112:46066) 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b098f2640 1 -- 172.21.15.112:0/3986759344 learned_addr learned my addr 172.21.15.112:0/3986759344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 msgr2=0x5647561da830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561da830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/3986759344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 msgr2=0x5647561dd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.795+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 0x5647561dd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0b0a0f3640 1 -- 172.21.15.112:0/3986759344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647561daed0 con 0x564755dfe2f0 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 0x5647561dd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0b0a0f3640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561dd8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b00004c40 tx=0x7f0b0002ab80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f2fb6b16e52cab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0aedffb640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0aedffb640 1 -- 172.21.15.112:0/3986759344 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00035070 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0aedffb640 1 -- 172.21.15.112:0/3986759344 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7f0b00004ef0 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af4004c00 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0aedffb640 1 -- 172.21.15.112:0/3986759344 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0b000050f0 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0b1320a740 10 client.26149 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56475624b440 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.796+0000 7f0aedffb640 1 -- 172.21.15.112:0/3986759344 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b000035f0 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0aedffb640 1 -- 172.21.15.112:0/3986759344 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0b00031560 con 0x564755dfe2f0 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b1320a740 10 client.26149 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b1320a740 10 client.26149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b1320a740 20 client.26149 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755e271b0 0x56475624dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b1320a740 4 client.26149 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:52:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f0af4004c00 con 0x564755e271b0 2024-04-25T16:52:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755e271b0 0x56475624dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.797+0000 7f0aedffb640 10 client.26149 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:28.798+0000 7f0b098f2640 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755e271b0 0x56475624dc90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b81e09d397b8dbaf server_cookie=624baf3a248d01d6 in_seq=0 out_seq=0 2024-04-25T16:52:29.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 10505 21.88 MB/sec warmup 34 sec latency 341.882 ms 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0aedffb640 1 -- 172.21.15.112:0/3986759344 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f0af4004c00 con 0x564755e271b0 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0aedffb640 10 client.26149 handle_command_reply: tid=0 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0b1320a740 1 client.26149 shutdown 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 msgr2=0x5647561dd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561dd8e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0b00004c40 tx=0x7f0b0002ab80 comp rx=0 tx=0).stop 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 shutdown_connections 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x564755e271b0 0x56475624dc90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.590+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564755dfe2f0 0x5647561dd8e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0aedffb640 0 client.26149 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564755e064d0 0x5647561da830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0b1320a740 1 --2- 172.21.15.112:0/3986759344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564755de5d80 0x5647561dd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 >> 172.21.15.112:0/3986759344 conn(0x564755e1b300 msgr2=0x5647561cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 shutdown_connections 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0b1320a740 1 -- 172.21.15.112:0/3986759344 wait complete. 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.591+0000 7f0b1320a740 20 client.26149 trim_cache size 0 max 16384 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:52:29.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:52:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.592+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f0af00afc50 msgr2=0x7f0af00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.592+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f0af00afc50 0x7f0af00b2160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0dd370 tx=0x7f0afc0ab040 comp rx=0 tx=0).stop 2024-04-25T16:52:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.592+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 msgr2=0x7f0b0419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:29.591 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.592+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 0x7f0b0419ef50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0afc00eb10 tx=0x7f0afc00c2d0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.594+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 shutdown_connections 2024-04-25T16:52:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.594+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f0af00afc50 0x7f0af00b2160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.594+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b04108f00 0x7f0b0419ef50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.594+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b04108a60 0x7f0b0419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.594+0000 7f0b0c357640 1 --2- 172.21.15.112:0/3659320616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b041031c0 0x7f0b0419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.594+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 >> 172.21.15.112:0/3659320616 conn(0x7f0b040781b0 msgr2=0x7f0b0406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.596+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 shutdown_connections 2024-04-25T16:52:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:29 smithi112 ceph-mon[29122]: pgmap v448: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.7 MiB/s wr, 256 op/s 2024-04-25T16:52:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:29.596+0000 7f0b0c357640 1 -- 172.21.15.112:0/3659320616 wait complete. 2024-04-25T16:52:29.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:29 smithi133 ceph-mon[32691]: pgmap v448: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.7 MiB/s wr, 256 op/s 2024-04-25T16:52:29.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:29 smithi116 ceph-mon[32712]: pgmap v448: 97 pgs: 97 active+clean; 301 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.7 MiB/s wr, 256 op/s 2024-04-25T16:52:30.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 10696 21.42 MB/sec warmup 35 sec latency 539.793 ms 2024-04-25T16:52:31.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 11062 21.42 MB/sec warmup 36 sec latency 220.737 ms 2024-04-25T16:52:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:31 smithi112 ceph-mon[29122]: pgmap v449: 97 pgs: 97 active+clean; 310 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.5 MiB/s wr, 312 op/s 2024-04-25T16:52:31.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:31 smithi133 ceph-mon[32691]: pgmap v449: 97 pgs: 97 active+clean; 310 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.5 MiB/s wr, 312 op/s 2024-04-25T16:52:31.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:31 smithi116 ceph-mon[32712]: pgmap v449: 97 pgs: 97 active+clean; 310 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 8.5 MiB/s wr, 312 op/s 2024-04-25T16:52:32.202 INFO:tasks.workunit.client.0.smithi112.stdout: 10 11600 21.33 MB/sec warmup 37 sec latency 223.140 ms 2024-04-25T16:52:33.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 12096 21.33 MB/sec warmup 38 sec latency 223.829 ms 2024-04-25T16:52:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:33 smithi112 ceph-mon[29122]: pgmap v450: 97 pgs: 97 active+clean; 310 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.7 MiB/s wr, 211 op/s 2024-04-25T16:52:33.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:33 smithi133 ceph-mon[32691]: pgmap v450: 97 pgs: 97 active+clean; 310 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.7 MiB/s wr, 211 op/s 2024-04-25T16:52:33.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:33 smithi116 ceph-mon[32712]: pgmap v450: 97 pgs: 97 active+clean; 310 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.7 MiB/s wr, 211 op/s 2024-04-25T16:52:34.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 12633 21.89 MB/sec warmup 39 sec latency 43.790 ms 2024-04-25T16:52:35.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 13058 22.20 MB/sec warmup 40 sec latency 155.833 ms 2024-04-25T16:52:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:35 smithi112 ceph-mon[29122]: pgmap v451: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.9 MiB/s wr, 503 op/s 2024-04-25T16:52:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:35 smithi133 ceph-mon[32691]: pgmap v451: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.9 MiB/s wr, 503 op/s 2024-04-25T16:52:35.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:35 smithi116 ceph-mon[32712]: pgmap v451: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.9 MiB/s wr, 503 op/s 2024-04-25T16:52:36.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 13338 22.05 MB/sec warmup 41 sec latency 93.690 ms 2024-04-25T16:52:37.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 13551 21.79 MB/sec warmup 42 sec latency 186.723 ms 2024-04-25T16:52:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:37 smithi112 ceph-mon[29122]: pgmap v452: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 378 op/s 2024-04-25T16:52:37.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:37 smithi133 ceph-mon[32691]: pgmap v452: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 378 op/s 2024-04-25T16:52:37.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:37 smithi116 ceph-mon[32712]: pgmap v452: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 378 op/s 2024-04-25T16:52:38.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 13713 21.34 MB/sec warmup 43 sec latency 241.189 ms 2024-04-25T16:52:39.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 13835 20.96 MB/sec warmup 44 sec latency 581.632 ms 2024-04-25T16:52:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:39 smithi112 ceph-mon[29122]: pgmap v453: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 378 op/s 2024-04-25T16:52:39.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:39 smithi133 ceph-mon[32691]: pgmap v453: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 378 op/s 2024-04-25T16:52:39.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:39 smithi116 ceph-mon[32712]: pgmap v453: 97 pgs: 97 active+clean; 293 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.1 MiB/s wr, 378 op/s 2024-04-25T16:52:40.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 14111 20.77 MB/sec warmup 45 sec latency 308.099 ms 2024-04-25T16:52:41.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 14529 20.85 MB/sec warmup 46 sec latency 192.369 ms 2024-04-25T16:52:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:41 smithi112 ceph-mon[29122]: pgmap v454: 97 pgs: 97 active+clean; 327 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 488 op/s 2024-04-25T16:52:41.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:41 smithi133 ceph-mon[32691]: pgmap v454: 97 pgs: 97 active+clean; 327 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 488 op/s 2024-04-25T16:52:41.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:41 smithi116 ceph-mon[32712]: pgmap v454: 97 pgs: 97 active+clean; 327 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.9 MiB/s wr, 488 op/s 2024-04-25T16:52:42.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 15196 20.72 MB/sec warmup 47 sec latency 180.000 ms 2024-04-25T16:52:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 -- 172.21.15.112:0/3607058135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 msgr2=0x7f38d4102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 --2- 172.21.15.112:0/3607058135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4102860 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f38cc009900 tx=0x7f38cc02a410 comp rx=0 tx=0).stop 2024-04-25T16:52:42.741 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 -- 172.21.15.112:0/3607058135 shutdown_connections 2024-04-25T16:52:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 --2- 172.21.15.112:0/3607058135 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 0x7f38d4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 --2- 172.21.15.112:0/3607058135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 0x7f38d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 --2- 172.21.15.112:0/3607058135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4102860 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 -- 172.21.15.112:0/3607058135 >> 172.21.15.112:0/3607058135 conn(0x7f38d40778b0 msgr2=0x7f38d4077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 -- 172.21.15.112:0/3607058135 shutdown_connections 2024-04-25T16:52:42.742 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.705+0000 7f38d9c51640 1 -- 172.21.15.112:0/3607058135 wait complete. 2024-04-25T16:52:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.706+0000 7f38d9c51640 1 Processor -- start 2024-04-25T16:52:42.755 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.706+0000 7f38d9c51640 1 -- start start 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d9c51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d9c51640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 0x7f38d4199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d9c51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 0x7f38d419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d9c51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41123b0 con 0x7f38d4102450 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d9c51640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d4112230 con 0x7f38d41074a0 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d9c51640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d4112530 con 0x7f38d41069e0 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 0x7f38d419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39030/0 (socket says 172.21.15.112:39030) 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 -- 172.21.15.112:0/2112379604 learned_addr learned my addr 172.21.15.112:0/2112379604 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 0x7f38d4199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 -- 172.21.15.112:0/2112379604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 msgr2=0x7f38d4199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:42.756 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 0x7f38d4199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 -- 172.21.15.112:0/2112379604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 msgr2=0x7f38d419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 0x7f38d419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 -- 172.21.15.112:0/2112379604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d419e4b0 con 0x7f38d4102450 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 0x7f38d4199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d3fff640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 0x7f38d419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.707+0000 7f38d37fe640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4199650 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005940 tx=0x7f38cc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.708+0000 7f38d0ff9640 1 -- 172.21.15.112:0/2112379604 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc03f030 con 0x7f38d4102450 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.708+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d419e740 con 0x7f38d4102450 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.708+0000 7f38d0ff9640 1 -- 172.21.15.112:0/2112379604 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38cc0044c0 con 0x7f38d4102450 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.708+0000 7f38d0ff9640 1 -- 172.21.15.112:0/2112379604 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc035070 con 0x7f38d4102450 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.708+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d419ec80 con 0x7f38d4102450 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 Processor -- start 2024-04-25T16:52:42.757 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 -- start start 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38d0ff9640 1 -- 172.21.15.112:0/2112379604 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f38cc042030 con 0x7f38d4102450 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cd94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cd92980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cd92f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56539ccf9460 con 0x56539c9ca2f0 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56539ccb3590 con 0x56539c9f31b0 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38e0be9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56539ccf5420 con 0x56539c9d24d0 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cd94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.710+0000 7f38d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cd92980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cd94090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:51130/0 (socket says 172.21.15.112:51130) 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 -- 172.21.15.112:0/907917255 learned_addr learned my addr 172.21.15.112:0/907917255 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cd92f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.758 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 -- 172.21.15.112:0/907917255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 msgr2=0x56539cd92f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cd92f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 -- 172.21.15.112:0/907917255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 msgr2=0x56539cd92980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cd92980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 -- 172.21.15.112:0/907917255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56539ccc00d0 con 0x56539c9f31b0 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cd92f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d0ff9640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f38bc0afc50 0x7f38bc0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d3fff640 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cd94090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f38c4011440 tx=0x7f38c4011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72fd5a49627173f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d37fe640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f38bc0afc50 0x7f38bc0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38b9ffb640 1 -- 172.21.15.112:0/907917255 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004020 con 0x56539c9f31b0 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38d0ff9640 1 -- 172.21.15.112:0/2112379604 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7f38cc0ff3e0 con 0x7f38d4102450 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38b9ffb640 1 -- 172.21.15.112:0/907917255 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c40041c0 con 0x56539c9f31b0 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.711+0000 7f38b9ffb640 1 -- 172.21.15.112:0/907917255 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c400f7a0 con 0x56539c9f31b0 2024-04-25T16:52:42.759 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 -- 172.21.15.112:0/907917255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 msgr2=0x56539cd94090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38d37fe640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f38bc0afc50 0x7f38bc0b2160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0fcb90 tx=0x7f38cc102f30 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cd94090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f38c4011440 tx=0x7f38c4011900 comp rx=0 tx=0).stop 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 -- 172.21.15.112:0/907917255 shutdown_connections 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cd92f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cd92980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 --2- 172.21.15.112:0/907917255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cd94090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 -- 172.21.15.112:0/907917255 >> 172.21.15.112:0/907917255 conn(0x56539c9e7300 msgr2=0x56539cd8f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 -- 172.21.15.112:0/907917255 shutdown_connections 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 -- 172.21.15.112:0/907917255 wait complete. 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.712+0000 7f38e0be9740 1 -- start start 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 Processor -- start 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cda7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.760 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x56539cda40a0 con 0x56539c9ca2f0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56539cc80b60 con 0x56539c9f31b0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56539cda4220 con 0x56539c9d24d0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38e0be9740 10 client.0 _ll_register_callbacks cb 0x56539cca52b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39044/0 (socket says 172.21.15.112:39044) 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 -- 172.21.15.112:0/456723669 learned_addr learned my addr 172.21.15.112:0/456723669 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cda7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 -- 172.21.15.112:0/456723669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 msgr2=0x56539cda1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 -- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 msgr2=0x56539cda7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cda7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.761 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.713+0000 7f38d3fff640 1 -- 172.21.15.112:0/456723669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56539ca0b210 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38d3fff640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda17e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f38c4005f60 tx=0x7f38c4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b983e4c61e592c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38b2ffd640 1 -- 172.21.15.112:0/456723669 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c402a020 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c40041c0 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38b2ffd640 1 -- 172.21.15.112:0/456723669 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7f38c4016070 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38b2ffd640 1 -- 172.21.15.112:0/456723669 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38c4032050 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 10 client.16575 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56539cad7620 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38b2ffd640 1 -- 172.21.15.112:0/456723669 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c40041c0 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38b2ffd640 1 -- 172.21.15.112:0/456723669 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f38c4004b20 con 0x56539c9ca2f0 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 10 client.16575 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:42.762 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 20 client.16575 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9b1d80 0x56539cdb3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 4 client.16575 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.714+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56539cd07070 con 0x56539c9b1d80 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.715+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9b1d80 0x56539cdb3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.715+0000 7f38b2ffd640 10 client.16575 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.715+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9b1d80 0x56539cdb3a70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a117eaa7ce13fc5 server_cookie=2a6980a63f54d0c9 in_seq=0 out_seq=0 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38b2ffd640 1 -- 172.21.15.112:0/456723669 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56539cd07070 con 0x56539c9b1d80 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38b2ffd640 10 client.16575 handle_command_reply: tid=0 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 client.16575 shutdown 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 msgr2=0x56539cda17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda17e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f38c4005f60 tx=0x7f38c4009ca0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 shutdown_connections 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9b1d80 0x56539cdb3a70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.763 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cda7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38b2ffd640 0 client.16575 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 --2- 172.21.15.112:0/456723669 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda17e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.717+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 >> 172.21.15.112:0/456723669 conn(0x56539c9e7300 msgr2=0x56539cd8e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.718+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 shutdown_connections 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.718+0000 7f38e0be9740 1 -- 172.21.15.112:0/456723669 wait complete. 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.718+0000 7f38e0be9740 20 client.16575 trim_cache size 0 max 16384 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.733+0000 7f38e0be9740 1 Processor -- start 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.733+0000 7f38e0be9740 1 -- start start 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cdb90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38e0be9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41b0290 con 0x56539c9ca2f0 2024-04-25T16:52:42.764 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38e0be9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41adb60 con 0x56539c9f31b0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38e0be9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41ab9d0 con 0x56539c9d24d0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39060/0 (socket says 172.21.15.112:39060) 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 -- 172.21.15.112:0/2820155823 learned_addr learned my addr 172.21.15.112:0/2820155823 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cdb90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 -- 172.21.15.112:0/2820155823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 msgr2=0x56539cda7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 -- 172.21.15.112:0/2820155823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 msgr2=0x56539cdb90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cdb90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 -- 172.21.15.112:0/2820155823 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56539cd8aa60 con 0x56539c9ca2f0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38d3fff640 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda1ca0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009bb0 tx=0x7f38c4011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf271348403eb2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:42.765 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38b97fa640 1 -- 172.21.15.112:0/2820155823 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c402a070 con 0x56539c9ca2f0 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38b97fa640 1 -- 172.21.15.112:0/2820155823 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c40045e0 con 0x56539c9ca2f0 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.734+0000 7f38b97fa640 1 -- 172.21.15.112:0/2820155823 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c401d030 con 0x56539c9ca2f0 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 -- 172.21.15.112:0/2820155823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 msgr2=0x56539cda1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda1ca0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009bb0 tx=0x7f38c4011900 comp rx=0 tx=0).stop 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 -- 172.21.15.112:0/2820155823 shutdown_connections 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9f31b0 0x56539cdb90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 --2- 172.21.15.112:0/2820155823 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9ca2f0 0x56539cda1ca0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 -- 172.21.15.112:0/2820155823 >> 172.21.15.112:0/2820155823 conn(0x56539c9e7300 msgr2=0x56539cda1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 -- 172.21.15.112:0/2820155823 shutdown_connections 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.735+0000 7f38e0be9740 1 -- 172.21.15.112:0/2820155823 wait complete. 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 -- start start 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 Processor -- start 2024-04-25T16:52:42.766 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9ca2f0 0x56539cda8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 0x56539ce13c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41ac020 con 0x56539c9f31b0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41ad490 con 0x56539c9ca2f0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38d41ac6f0 con 0x56539c9d24d0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38e0be9740 10 client.0 _ll_register_callbacks cb 0x7f38cc102e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9ca2f0 0x56539cda8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 0x56539ce13c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.736+0000 7f38d2ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 0x56539ce13c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:39076/0 (socket says 172.21.15.112:39076) 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 -- 172.21.15.112:0/1550613936 learned_addr learned my addr 172.21.15.112:0/1550613936 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 -- 172.21.15.112:0/1550613936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 msgr2=0x56539cda9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.767 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 -- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9ca2f0 msgr2=0x56539cda8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9ca2f0 0x56539cda8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 -- 172.21.15.112:0/1550613936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56539ce141d0 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38d3fff640 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 0x56539ce13c40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f38c4003b20 tx=0x7f38c4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8f511b17f1f72e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38b2ffd640 1 -- 172.21.15.112:0/1550613936 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c402a070 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38b2ffd640 1 -- 172.21.15.112:0/1550613936 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7f38c4004790 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c40045e0 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38b2ffd640 1 -- 172.21.15.112:0/1550613936 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f38c404e050 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38e0be9740 10 client.16581 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.737+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56539ce1b500 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38b2ffd640 1 -- 172.21.15.112:0/1550613936 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c40045e0 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38b2ffd640 1 -- 172.21.15.112:0/1550613936 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f38c4044070 con 0x56539c9f31b0 2024-04-25T16:52:42.768 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38e0be9740 10 client.16581 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38e0be9740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38e0be9740 20 client.16581 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38e0be9740 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9871c0 0x56539ce1db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38e0be9740 4 client.16581 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56539cd00ee0 con 0x56539c9871c0 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9871c0 0x56539ce1db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38b2ffd640 10 client.16581 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:42.769 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:42.738+0000 7f38d2ffd640 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9871c0 0x56539ce1db40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d29d95dd880abf server_cookie=4d4b3e25588e7d39 in_seq=0 out_seq=0 2024-04-25T16:52:43.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 15374 20.51 MB/sec warmup 48 sec latency 284.129 ms 2024-04-25T16:52:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.446+0000 7f38b2ffd640 1 -- 172.21.15.112:0/1550613936 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56539cd00ee0 con 0x56539c9871c0 2024-04-25T16:52:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.446+0000 7f38b2ffd640 10 client.16581 handle_command_reply: tid=0 2024-04-25T16:52:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.446+0000 7f38e0be9740 1 client.16581 shutdown 2024-04-25T16:52:43.445 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 msgr2=0x56539ce13c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 0x56539ce13c40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f38c4003b20 tx=0x7f38c4004570 comp rx=0 tx=0).stop 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 shutdown_connections 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x56539c9871c0 0x56539ce1db40 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56539c9d24d0 0x56539cda9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56539c9f31b0 0x56539ce13c40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38b2ffd640 0 client.16581 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 --2- 172.21.15.112:0/1550613936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56539c9ca2f0 0x56539cda8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 >> 172.21.15.112:0/1550613936 conn(0x56539c9e7300 msgr2=0x56539cd8eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 shutdown_connections 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 1 -- 172.21.15.112:0/1550613936 wait complete. 2024-04-25T16:52:43.446 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.447+0000 7f38e0be9740 20 client.16581 trim_cache size 0 max 16384 2024-04-25T16:52:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:52:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:52:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:52:43.447 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:52:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f38bc0afc50 msgr2=0x7f38bc0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f38bc0afc50 0x7f38bc0b2160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0fcb90 tx=0x7f38cc102f30 comp rx=0 tx=0).stop 2024-04-25T16:52:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 msgr2=0x7f38d4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4199650 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f38cc005940 tx=0x7f38cc0040c0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.448 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 shutdown_connections 2024-04-25T16:52:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f38bc0afc50 0x7f38bc0b2160 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38d41069e0 0x7f38d4199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d41074a0 0x7f38d419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 --2- 172.21.15.112:0/2112379604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4102450 0x7f38d4199650 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:43.449 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 >> 172.21.15.112:0/2112379604 conn(0x7f38d40778b0 msgr2=0x7f38d4103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 shutdown_connections 2024-04-25T16:52:43.450 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:43.448+0000 7f38d9c51640 1 -- 172.21.15.112:0/2112379604 wait complete. 2024-04-25T16:52:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:43 smithi112 ceph-mon[29122]: pgmap v455: 97 pgs: 97 active+clean; 327 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-04-25T16:52:43.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:43 smithi133 ceph-mon[32691]: pgmap v455: 97 pgs: 97 active+clean; 327 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-04-25T16:52:43.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:43 smithi116 ceph-mon[32712]: pgmap v455: 97 pgs: 97 active+clean; 327 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-04-25T16:52:44.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 15724 20.77 MB/sec warmup 49 sec latency 418.701 ms 2024-04-25T16:52:45.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 16284 21.24 MB/sec warmup 50 sec latency 63.351 ms 2024-04-25T16:52:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:45 smithi112 ceph-mon[29122]: pgmap v456: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 748 op/s 2024-04-25T16:52:45.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:45 smithi133 ceph-mon[32691]: pgmap v456: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 748 op/s 2024-04-25T16:52:45.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:45 smithi116 ceph-mon[32712]: pgmap v456: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.7 MiB/s wr, 748 op/s 2024-04-25T16:52:46.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 16601 21.15 MB/sec warmup 51 sec latency 207.075 ms 2024-04-25T16:52:47.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 16825 21.02 MB/sec warmup 52 sec latency 192.861 ms 2024-04-25T16:52:47.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:47 smithi112 ceph-mon[29122]: pgmap v457: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-04-25T16:52:47.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:47 smithi133 ceph-mon[32691]: pgmap v457: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-04-25T16:52:47.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:47 smithi116 ceph-mon[32712]: pgmap v457: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-04-25T16:52:48.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 17004 20.75 MB/sec warmup 53 sec latency 191.663 ms 2024-04-25T16:52:49.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 17122 20.39 MB/sec warmup 54 sec latency 411.828 ms 2024-04-25T16:52:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:49 smithi112 ceph-mon[29122]: pgmap v458: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-04-25T16:52:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:49 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:52:49.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:49 smithi133 ceph-mon[32691]: pgmap v458: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-04-25T16:52:49.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:49 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:52:49.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:49 smithi116 ceph-mon[32712]: pgmap v458: 97 pgs: 97 active+clean; 298 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.5 MiB/s wr, 456 op/s 2024-04-25T16:52:49.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:49 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:52:50.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 17388 20.21 MB/sec warmup 55 sec latency 281.279 ms 2024-04-25T16:52:50.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:52:50.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:52:50.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:52:50.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:52:50.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:52:50.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:52:50.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:52:50.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:52:50.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:52:51.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 17768 20.36 MB/sec warmup 56 sec latency 103.063 ms 2024-04-25T16:52:51.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:51 smithi133 ceph-mon[32691]: pgmap v459: 97 pgs: 97 active+clean; 318 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 564 op/s 2024-04-25T16:52:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:51 smithi116 ceph-mon[32712]: pgmap v459: 97 pgs: 97 active+clean; 318 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 564 op/s 2024-04-25T16:52:51.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:51 smithi112 ceph-mon[29122]: pgmap v459: 97 pgs: 97 active+clean; 318 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 7.8 MiB/s wr, 564 op/s 2024-04-25T16:52:52.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 18214 20.12 MB/sec warmup 57 sec latency 67.254 ms 2024-04-25T16:52:53.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 18712 20.09 MB/sec warmup 58 sec latency 196.973 ms 2024-04-25T16:52:53.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:53 smithi133 ceph-mon[32691]: pgmap v460: 97 pgs: 97 active+clean; 318 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-04-25T16:52:53.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:53 smithi116 ceph-mon[32712]: pgmap v460: 97 pgs: 97 active+clean; 318 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-04-25T16:52:53.845 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:53 smithi112 ceph-mon[29122]: pgmap v460: 97 pgs: 97 active+clean; 318 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 454 op/s 2024-04-25T16:52:54.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 19083 20.34 MB/sec warmup 59 sec latency 248.410 ms 2024-04-25T16:52:55.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 19593 20.50 MB/sec warmup 60 sec latency 185.935 ms 2024-04-25T16:52:55.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:55 smithi133 ceph-mon[32691]: pgmap v461: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.6 MiB/s wr, 585 op/s 2024-04-25T16:52:55.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:55 smithi116 ceph-mon[32712]: pgmap v461: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.6 MiB/s wr, 585 op/s 2024-04-25T16:52:55.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:55 smithi112 ceph-mon[29122]: pgmap v461: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.6 MiB/s wr, 585 op/s 2024-04-25T16:52:56.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 19883 20.58 MB/sec warmup 61 sec latency 227.162 ms 2024-04-25T16:52:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 -- 172.21.15.112:0/3037121141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c1032e0 msgr2=0x7efd4c1036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 --2- 172.21.15.112:0/3037121141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c1032e0 0x7efd4c1036f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009900 tx=0x7efd3c02a410 comp rx=0 tx=0).stop 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 -- 172.21.15.112:0/3037121141 shutdown_connections 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 --2- 172.21.15.112:0/3037121141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd4c10d460 0x7efd4c10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 --2- 172.21.15.112:0/3037121141 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c1032e0 0x7efd4c1036f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 --2- 172.21.15.112:0/3037121141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c107970 0x7efd4c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.548+0000 7efd54092640 1 -- 172.21.15.112:0/3037121141 >> 172.21.15.112:0/3037121141 conn(0x7efd4c0fd8a0 msgr2=0x7efd4c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 -- 172.21.15.112:0/3037121141 shutdown_connections 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 -- 172.21.15.112:0/3037121141 wait complete. 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 Processor -- start 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 -- start start 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd4c1032e0 0x7efd4c104550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 0x7efd4c104a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c10d460 0x7efd4c104fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c112c80 con 0x7efd4c107970 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c112b00 con 0x7efd4c10d460 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.549+0000 7efd54092640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c112e00 con 0x7efd4c1032e0 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 0x7efd4c104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 0x7efd4c104a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56320/0 (socket says 172.21.15.112:56320) 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 -- 172.21.15.112:0/2054222963 learned_addr learned my addr 172.21.15.112:0/2054222963 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5262f640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c10d460 0x7efd4c104fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd51e2e640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd4c1032e0 0x7efd4c104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 -- 172.21.15.112:0/2054222963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd4c1032e0 msgr2=0x7efd4c104550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd4c1032e0 0x7efd4c104550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 -- 172.21.15.112:0/2054222963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c10d460 msgr2=0x7efd4c104fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c10d460 0x7efd4c104fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.550 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 -- 172.21.15.112:0/2054222963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4c1a8b10 con 0x7efd4c107970 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5262f640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c10d460 0x7efd4c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd5162d640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 0x7efd4c104a90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efd3c006010 tx=0x7efd3c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd42ffd640 1 -- 172.21.15.112:0/2054222963 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c03f030 con 0x7efd4c107970 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4c1a8da0 con 0x7efd4c107970 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd42ffd640 1 -- 172.21.15.112:0/2054222963 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c003aa0 con 0x7efd4c107970 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd42ffd640 1 -- 172.21.15.112:0/2054222963 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c035070 con 0x7efd4c107970 2024-04-25T16:52:56.551 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.550+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd4c1a91d0 con 0x7efd4c107970 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.553+0000 7efd42ffd640 1 -- 172.21.15.112:0/2054222963 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7efd3c042030 con 0x7efd4c107970 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.553+0000 7efd5b014740 1 Processor -- start 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd5b014740 1 -- start start 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd5b014740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd5398af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd5b014740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd5399030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd42ffd640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efd380afc50 0x7efd380b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd5b014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd5399570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd5399030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd5399030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:36750/0 (socket says 172.21.15.112:36750) 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd5262f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd5398af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.554+0000 7efd5262f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd5398af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:58768/0 (socket says 172.21.15.112:58768) 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 -- 172.21.15.112:0/1147577533 learned_addr learned my addr 172.21.15.112:0/1147577533 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd5162d640 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd5399570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fd52ff460 con 0x559fd4fd84d0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559fd52b9590 con 0x559fd4fd02f0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559fd52fb420 con 0x559fd4ff91b0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd42ffd640 1 -- 172.21.15.112:0/2054222963 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7efd3c0ff4e0 con 0x7efd4c107970 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd5262f640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efd380afc50 0x7efd380b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 -- 172.21.15.112:0/1147577533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 msgr2=0x559fd5398af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd5398af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 -- 172.21.15.112:0/1147577533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 msgr2=0x559fd5399570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd5399570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 -- 172.21.15.112:0/1147577533 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd52c60d0 con 0x559fd4fd02f0 2024-04-25T16:52:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd51e2e640 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd5399030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd48009c80 tx=0x7efd480117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7c776d6d1b00ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd5262f640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efd380afc50 0x7efd380b2160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x559fd5395c60 tx=0x7efd44009650 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd37fff640 1 -- 172.21.15.112:0/1147577533 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48012450 con 0x559fd4fd02f0 2024-04-25T16:52:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd37fff640 1 -- 172.21.15.112:0/1147577533 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd48012aa0 con 0x559fd4fd02f0 2024-04-25T16:52:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.555+0000 7efd37fff640 1 -- 172.21.15.112:0/1147577533 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4800fb00 con 0x559fd4fd02f0 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 msgr2=0x559fd5399030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd5399030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd48009c80 tx=0x7efd480117b0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 shutdown_connections 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd5399570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd5399030 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 --2- 172.21.15.112:0/1147577533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd5398af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 >> 172.21.15.112:0/1147577533 conn(0x559fd4fed300 msgr2=0x559fd53952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 shutdown_connections 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.556+0000 7efd5b014740 1 -- 172.21.15.112:0/1147577533 wait complete. 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 -- start start 2024-04-25T16:52:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 Processor -- start 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd53a3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd53a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x559fd53a5c20 con 0x559fd4fd84d0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559fd5286b60 con 0x559fd4fd02f0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559fd53a5da0 con 0x559fd4ff91b0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.557+0000 7efd5b014740 10 client.0 _ll_register_callbacks cb 0x559fd52ab2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd53a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53a31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:36764/0 (socket says 172.21.15.112:36764) 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd51e2e640 1 -- 172.21.15.112:0/2742356206 learned_addr learned my addr 172.21.15.112:0/2742356206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5262f640 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd53a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 -- 172.21.15.112:0/2742356206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 msgr2=0x559fd53a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd53a9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 -- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 msgr2=0x559fd53a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53a31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 -- 172.21.15.112:0/2742356206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd5011210 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5162d640 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd53a3720 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0fcbe0 tx=0x7efd3c0c6f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7defa014065d2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd357fa640 1 -- 172.21.15.112:0/2742356206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c049070 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd357fa640 1 -- 172.21.15.112:0/2742356206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7efd3c0fce60 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd48012aa0 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd357fa640 1 -- 172.21.15.112:0/2742356206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efd3c0c8760 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5b014740 10 client.16593 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.558+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559fd50dd620 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd357fa640 1 -- 172.21.15.112:0/2742356206 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd48012aa0 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd357fa640 1 -- 172.21.15.112:0/2742356206 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efd3c0c7890 con 0x559fd4fd84d0 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd5b014740 10 client.16593 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd5b014740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd5b014740 20 client.16593 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd5b014740 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4fb7d80 0x559fd53b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd5b014740 4 client.16593 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fd530d070 con 0x559fd4fb7d80 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd51e2e640 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4fb7d80 0x559fd53b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd357fa640 10 client.16593 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.559+0000 7efd51e2e640 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4fb7d80 0x559fd53b5790 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1eadb413c18fb30 server_cookie=b2e120c4e8da896 in_seq=0 out_seq=0 2024-04-25T16:52:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd357fa640 1 -- 172.21.15.112:0/2742356206 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559fd530d070 con 0x559fd4fb7d80 2024-04-25T16:52:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd357fa640 10 client.16593 handle_command_reply: tid=0 2024-04-25T16:52:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 client.16593 shutdown 2024-04-25T16:52:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 msgr2=0x559fd53a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd53a3720 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0fcbe0 tx=0x7efd3c0c6f80 comp rx=0 tx=0).stop 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 shutdown_connections 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4fb7d80 0x559fd53b5790 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4ff91b0 0x559fd53a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd84d0 0x559fd53a3720 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd357fa640 0 client.16593 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 --2- 172.21.15.112:0/2742356206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53a31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 >> 172.21.15.112:0/2742356206 conn(0x559fd4fed300 msgr2=0x559fd5398920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 shutdown_connections 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 1 -- 172.21.15.112:0/2742356206 wait complete. 2024-04-25T16:52:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.562+0000 7efd5b014740 20 client.16593 trim_cache size 0 max 16384 2024-04-25T16:52:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.577+0000 7efd5b014740 1 Processor -- start 2024-04-25T16:52:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.577+0000 7efd5b014740 1 -- start start 2024-04-25T16:52:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5b014740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5b014740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5b014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 0x559fd539c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5b014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1b4640 con 0x559fd4ff91b0 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5b014740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1b1f10 con 0x559fd4fd02f0 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5b014740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1afd80 con 0x559fd4fd84d0 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 0x559fd539c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 0x559fd539c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56342/0 (socket says 172.21.15.112:56342) 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 -- 172.21.15.112:0/733071354 learned_addr learned my addr 172.21.15.112:0/733071354 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5162d640 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 -- 172.21.15.112:0/733071354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 msgr2=0x559fd53bb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53bb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 -- 172.21.15.112:0/733071354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 msgr2=0x559fd53baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 -- 172.21.15.112:0/733071354 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fd53a3ef0 con 0x559fd4ff91b0 2024-04-25T16:52:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd5262f640 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 0x559fd539c7b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efd44017f00 tx=0x7efd44009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61f360abdb05a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.578+0000 7efd377fe640 1 -- 172.21.15.112:0/733071354 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd44014040 con 0x559fd4ff91b0 2024-04-25T16:52:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd377fe640 1 -- 172.21.15.112:0/733071354 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4402b070 con 0x559fd4ff91b0 2024-04-25T16:52:56.578 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd377fe640 1 -- 172.21.15.112:0/733071354 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd440195e0 con 0x559fd4ff91b0 2024-04-25T16:52:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 -- 172.21.15.112:0/733071354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 msgr2=0x559fd539c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 0x559fd539c7b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efd44017f00 tx=0x7efd44009e80 comp rx=0 tx=0).stop 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 -- 172.21.15.112:0/733071354 shutdown_connections 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4ff91b0 0x559fd539c7b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 --2- 172.21.15.112:0/733071354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4fd02f0 0x559fd53baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 -- 172.21.15.112:0/733071354 >> 172.21.15.112:0/733071354 conn(0x559fd4fed300 msgr2=0x559fd53a18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 -- 172.21.15.112:0/733071354 shutdown_connections 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.579+0000 7efd5b014740 1 -- 172.21.15.112:0/733071354 wait complete. 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 -- start start 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 Processor -- start 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 0x559fd53af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4ff91b0 0x559fd53ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1b03d0 con 0x559fd4fd02f0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1b1840 con 0x559fd4ff91b0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd4c1b0aa0 con 0x559fd4fd84d0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5b014740 10 client.0 _ll_register_callbacks cb 0x7efd3c0c3510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 0x559fd53af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd51e2e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 0x559fd53af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56358/0 (socket says 172.21.15.112:56358) 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd51e2e640 1 -- 172.21.15.112:0/1783853167 learned_addr learned my addr 172.21.15.112:0/1783853167 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5262f640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4ff91b0 0x559fd53ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.580+0000 7efd5162d640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd51e2e640 1 -- 172.21.15.112:0/1783853167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 msgr2=0x559fd53acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd51e2e640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd51e2e640 1 -- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4ff91b0 msgr2=0x559fd53ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:52:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd51e2e640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4ff91b0 0x559fd53ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd51e2e640 1 -- 172.21.15.112:0/1783853167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fd53ad5c0 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd5262f640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4ff91b0 0x559fd53ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd51e2e640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 0x559fd53af7d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efd48005fd0 tx=0x7efd480058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e57419c6bdee44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd357fa640 1 -- 172.21.15.112:0/1783853167 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd48033060 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd357fa640 1 -- 172.21.15.112:0/1783853167 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8600+0+0 (secure 0 0 0) 0x7efd480052b0 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd4402b070 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd357fa640 1 -- 172.21.15.112:0/1783853167 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efd480054b0 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd5b014740 10 client.16599 fetch_fsmap learned FSMap version 39 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559fd541d6e0 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.581+0000 7efd357fa640 1 -- 172.21.15.112:0/1783853167 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd4402b070 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd357fa640 1 -- 172.21.15.112:0/1783853167 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efd48003680 con 0x559fd4fd02f0 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5b014740 10 client.16599 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:52:56.584 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5b014740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5b014740 20 client.16599 populate_metadata read hostname 'smithi112' 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5b014740 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4f8d1c0 0x559fd541fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5b014740 4 client.16599 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fd5306ee0 con 0x559fd4f8d1c0 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5162d640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4f8d1c0 0x559fd541fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd357fa640 10 client.16599 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:56.582+0000 7efd5162d640 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4f8d1c0 0x559fd541fc40 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1548879e44958e3 server_cookie=4d8fcb16007e4c84 in_seq=0 out_seq=0 2024-04-25T16:52:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd357fa640 1 -- 172.21.15.112:0/1783853167 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fd5306ee0 con 0x559fd4f8d1c0 2024-04-25T16:52:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd357fa640 10 client.16599 handle_command_reply: tid=0 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 client.16599 shutdown 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 msgr2=0x559fd53af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 0x559fd53af7d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efd48005fd0 tx=0x7efd480058e0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 shutdown_connections 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x559fd4f8d1c0 0x559fd541fc40 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559fd4fd84d0 0x559fd53acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559fd4ff91b0 0x559fd53ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 --2- 172.21.15.112:0/1783853167 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x559fd4fd02f0 0x559fd53af7d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 >> 172.21.15.112:0/1783853167 conn(0x559fd4fed300 msgr2=0x559fd539c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.137+0000 7efd357fa640 0 client.16599 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 shutdown_connections 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd5b014740 1 -- 172.21.15.112:0/1783853167 wait complete. 2024-04-25T16:52:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd5b014740 20 client.16599 trim_cache size 0 max 16384 2024-04-25T16:52:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:52:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:52:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:52:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efd380afc50 msgr2=0x7efd380b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efd380afc50 0x7efd380b2160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x559fd5395c60 tx=0x7efd44009650 comp rx=0 tx=0).stop 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 msgr2=0x7efd4c104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 0x7efd4c104a90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efd3c006010 tx=0x7efd3c004060 comp rx=0 tx=0).stop 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 shutdown_connections 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7efd380afc50 0x7efd380b2160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd4c10d460 0x7efd4c104fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efd4c107970 0x7efd4c104a90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 --2- 172.21.15.112:0/2054222963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd4c1032e0 0x7efd4c104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:52:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 >> 172.21.15.112:0/2054222963 conn(0x7efd4c0fd8a0 msgr2=0x7efd4c101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:52:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.138+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 shutdown_connections 2024-04-25T16:52:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:52:57.139+0000 7efd54092640 1 -- 172.21.15.112:0/2054222963 wait complete. 2024-04-25T16:52:57.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20042 20.38 MB/sec warmup 62 sec latency 414.748 ms 2024-04-25T16:52:57.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:57 smithi133 ceph-mon[32691]: pgmap v462: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-25T16:52:57.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:57 smithi116 ceph-mon[32712]: pgmap v462: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-25T16:52:57.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:57 smithi112 ceph-mon[29122]: pgmap v462: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-25T16:52:58.203 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20237 20.19 MB/sec warmup 63 sec latency 284.685 ms 2024-04-25T16:52:58.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:52:58 smithi116 ceph-mon[32712]: pgmap v463: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-25T16:52:58.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:52:58 smithi112 ceph-mon[29122]: pgmap v463: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-25T16:52:58.948 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:52:58 smithi133 ceph-mon[32691]: pgmap v463: 97 pgs: 97 active+clean; 315 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.0 MiB/s wr, 238 op/s 2024-04-25T16:52:59.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20376 19.91 MB/sec warmup 64 sec latency 343.164 ms 2024-04-25T16:53:00.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20584 19.73 MB/sec warmup 65 sec latency 244.385 ms 2024-04-25T16:53:01.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:00 smithi112 ceph-mon[29122]: pgmap v464: 97 pgs: 97 active+clean; 314 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 531 op/s 2024-04-25T16:53:01.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:00 smithi133 ceph-mon[32691]: pgmap v464: 97 pgs: 97 active+clean; 314 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 531 op/s 2024-04-25T16:53:01.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20745 19.51 MB/sec warmup 66 sec latency 591.335 ms 2024-04-25T16:53:01.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:00 smithi116 ceph-mon[32712]: pgmap v464: 97 pgs: 97 active+clean; 314 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.5 MiB/s wr, 531 op/s 2024-04-25T16:53:02.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20774 19.23 MB/sec warmup 67 sec latency 1262.852 ms 2024-04-25T16:53:03.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:02 smithi112 ceph-mon[29122]: pgmap v465: 97 pgs: 97 active+clean; 314 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-04-25T16:53:03.199 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:02 smithi133 ceph-mon[32691]: pgmap v465: 97 pgs: 97 active+clean; 314 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-04-25T16:53:03.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20774 18.94 MB/sec warmup 68 sec latency 1263.802 ms 2024-04-25T16:53:03.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:02 smithi116 ceph-mon[32712]: pgmap v465: 97 pgs: 97 active+clean; 314 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.2 MiB/s wr, 423 op/s 2024-04-25T16:53:04.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 20800 18.68 MB/sec warmup 69 sec latency 2197.588 ms 2024-04-25T16:53:05.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:04 smithi112 ceph-mon[29122]: pgmap v466: 97 pgs: 97 active+clean; 311 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 444 op/s 2024-04-25T16:53:05.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:04 smithi133 ceph-mon[32691]: pgmap v466: 97 pgs: 97 active+clean; 311 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 444 op/s 2024-04-25T16:53:05.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 21082 18.68 MB/sec warmup 70 sec latency 118.833 ms 2024-04-25T16:53:05.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:04 smithi116 ceph-mon[32712]: pgmap v466: 97 pgs: 97 active+clean; 311 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 444 op/s 2024-04-25T16:53:06.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 21291 18.53 MB/sec warmup 71 sec latency 225.725 ms 2024-04-25T16:53:07.060 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:06 smithi116 ceph-mon[32712]: pgmap v467: 97 pgs: 97 active+clean; 297 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-04-25T16:53:07.060 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:06 smithi116 ceph-mon[32712]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T16:53:07.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:06 smithi112 ceph-mon[29122]: pgmap v467: 97 pgs: 97 active+clean; 297 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-04-25T16:53:07.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:06 smithi112 ceph-mon[29122]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T16:53:07.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:06 smithi133 ceph-mon[32691]: pgmap v467: 97 pgs: 97 active+clean; 297 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.3 MiB/s wr, 374 op/s 2024-04-25T16:53:07.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:06 smithi133 ceph-mon[32691]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T16:53:07.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 21550 18.47 MB/sec warmup 72 sec latency 435.809 ms 2024-04-25T16:53:08.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 21897 18.38 MB/sec warmup 73 sec latency 407.890 ms 2024-04-25T16:53:09.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:08 smithi112 ceph-mon[29122]: pgmap v469: 97 pgs: 97 active+clean; 297 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-04-25T16:53:09.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:08 smithi133 ceph-mon[32691]: pgmap v469: 97 pgs: 97 active+clean; 297 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-04-25T16:53:09.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 22025 18.19 MB/sec warmup 74 sec latency 317.044 ms 2024-04-25T16:53:09.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:08 smithi116 ceph-mon[32712]: pgmap v469: 97 pgs: 97 active+clean; 297 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 449 op/s 2024-04-25T16:53:10.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 22350 18.35 MB/sec warmup 75 sec latency 213.469 ms 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 -- 172.21.15.112:0/2941277692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 msgr2=0x7f63c8101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 --2- 172.21.15.112:0/2941277692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c8101b60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009ca0 tx=0x7f63b802a8c0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 -- 172.21.15.112:0/2941277692 shutdown_connections 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 --2- 172.21.15.112:0/2941277692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63c810e5a0 0x7f63c81109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 --2- 172.21.15.112:0/2941277692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63c81020a0 0x7f63c81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 --2- 172.21.15.112:0/2941277692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c8101b60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 -- 172.21.15.112:0/2941277692 >> 172.21.15.112:0/2941277692 conn(0x7f63c80778b0 msgr2=0x7f63c8077cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 -- 172.21.15.112:0/2941277692 shutdown_connections 2024-04-25T16:53:10.244 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.244+0000 7f63d0427640 1 -- 172.21.15.112:0/2941277692 wait complete. 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 Processor -- start 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 -- start start 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63c81020a0 0x7f63c819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63c810e5a0 0x7f63c819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113d60 con 0x7f63c8109bf0 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113be0 con 0x7f63c81020a0 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63d0427640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8113ee0 con 0x7f63c810e5a0 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63ce1c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63c81020a0 0x7f63c819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c819ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45732/0 (socket says 172.21.15.112:45732) 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 -- 172.21.15.112:0/1754634080 learned_addr learned my addr 172.21.15.112:0/1754634080 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63c810e5a0 0x7f63c819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.245 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 -- 172.21.15.112:0/1754634080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63c810e5a0 msgr2=0x7f63c819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63c810e5a0 0x7f63c819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 -- 172.21.15.112:0/1754634080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63c81020a0 msgr2=0x7f63c819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63c81020a0 0x7f63c819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.245+0000 7f63cd9c2640 1 -- 172.21.15.112:0/1754634080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c819f5e0 con 0x7f63c8109bf0 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.246+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c819ace0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00e9c0 tx=0x7f63bc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.246+0000 7f63c77fe640 1 -- 172.21.15.112:0/1754634080 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00cd80 con 0x7f63c8109bf0 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.246+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c819f8d0 con 0x7f63c8109bf0 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.246+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81a7250 con 0x7f63c8109bf0 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.246+0000 7f63c77fe640 1 -- 172.21.15.112:0/1754634080 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc0054d0 con 0x7f63c8109bf0 2024-04-25T16:53:10.246 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.246+0000 7f63c77fe640 1 -- 172.21.15.112:0/1754634080 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc004e40 con 0x7f63c8109bf0 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 Processor -- start 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 -- start start 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63c77fe640 1 -- 172.21.15.112:0/1754634080 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f63bc021440 con 0x7f63c8109bf0 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893337caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fb42f0 0x55893337d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fbc4d0 0x55893337d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589332e3460 con 0x558932fdd1b0 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55893329d590 con 0x558932fb42f0 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63d73ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589332df420 con 0x558932fbc4d0 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63ce9c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893337caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63ce9c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893337caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45738/0 (socket says 172.21.15.112:45738) 2024-04-25T16:53:10.252 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63ce9c4640 1 -- 172.21.15.112:0/1190552127 learned_addr learned my addr 172.21.15.112:0/1190552127 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fbc4d0 0x55893337d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.249+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fb42f0 0x55893337d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce9c4640 1 -- 172.21.15.112:0/1190552127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fbc4d0 msgr2=0x55893337d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fbc4d0 0x55893337d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce9c4640 1 -- 172.21.15.112:0/1190552127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fb42f0 msgr2=0x55893337d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fb42f0 0x55893337d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce9c4640 1 -- 172.21.15.112:0/1190552127 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589332aa0d0 con 0x558932fdd1b0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893337caf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009c20 tx=0x7f63c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37abe7ed0f600de9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63c77fe640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f63b00afc50 0x7f63b00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f63b00afc50 0x7f63b00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63c4bfe640 1 -- 172.21.15.112:0/1190552127 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c001e070 con 0x558932fdd1b0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63c4bfe640 1 -- 172.21.15.112:0/1190552127 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63c0019ae0 con 0x558932fdd1b0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63c77fe640 1 -- 172.21.15.112:0/1754634080 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8644+0+0 (secure 0 0 0) 0x7f63bc0df390 con 0x7f63c8109bf0 2024-04-25T16:53:10.253 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.250+0000 7f63c4bfe640 1 -- 172.21.15.112:0/1190552127 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c000e920 con 0x558932fdd1b0 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 -- 172.21.15.112:0/1190552127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 msgr2=0x55893337caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f63b00afc50 0x7f63b00b2160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000f80 tx=0x7f63b8009360 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893337caf0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009c20 tx=0x7f63c0011810 comp rx=0 tx=0).stop 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 -- 172.21.15.112:0/1190552127 shutdown_connections 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fbc4d0 0x55893337d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fb42f0 0x55893337d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1190552127 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893337caf0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 -- 172.21.15.112:0/1190552127 >> 172.21.15.112:0/1190552127 conn(0x558932fd1300 msgr2=0x5589333792a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 -- 172.21.15.112:0/1190552127 shutdown_connections 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 -- 172.21.15.112:0/1190552127 wait complete. 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.251+0000 7f63d73ba740 1 -- start start 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 Processor -- start 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933387170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.254 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333876b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893338d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x558933389c20 con 0x558932fdd1b0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55893326ab60 con 0x558932fbc4d0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558933389da0 con 0x558932fb42f0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63d73ba740 10 client.0 _ll_register_callbacks cb 0x55893328f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893338d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce1c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933387170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce1c3640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933387170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:40994/0 (socket says 172.21.15.112:40994) 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893338d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45750/0 (socket says 172.21.15.112:45750) 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce1c3640 1 -- 172.21.15.112:0/2920520587 learned_addr learned my addr 172.21.15.112:0/2920520587 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63cd9c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333876b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 -- 172.21.15.112:0/2920520587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 msgr2=0x558933387170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933387170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 -- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 msgr2=0x5589333876b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.255 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333876b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 -- 172.21.15.112:0/2920520587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558932ff5210 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933387170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893338d110 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009bf0 tx=0x7f63c00098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59273ddcda28d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.252+0000 7f63adffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63adffb640 1 -- 172.21.15.112:0/2920520587 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0029020 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c0019ae0 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63adffb640 1 -- 172.21.15.112:0/2920520587 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8644+0+0 (secure 0 0 0) 0x7f63c0005450 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63adffb640 1 -- 172.21.15.112:0/2920520587 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63c001e050 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 10 client.16611 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589330c1620 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63adffb640 1 -- 172.21.15.112:0/2920520587 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63c0019ae0 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63adffb640 1 -- 172.21.15.112:0/2920520587 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f63c000e8c0 con 0x558932fdd1b0 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 10 client.16611 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:10.256 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 20 client.16611 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f9bd80 0x558933399630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 4 client.16611 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589332f1070 con 0x558932f9bd80 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.253+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f9bd80 0x558933399630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.254+0000 7f63adffb640 10 client.16611 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:10.257 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.254+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f9bd80 0x558933399630 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=477583ddca73c66c server_cookie=c41a6bf534e4ed76 in_seq=0 out_seq=0 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63adffb640 1 -- 172.21.15.112:0/2920520587 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589332f1070 con 0x558932f9bd80 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63adffb640 10 client.16611 handle_command_reply: tid=0 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 client.16611 shutdown 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 msgr2=0x55893338d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893338d110 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009bf0 tx=0x7f63c00098d0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 shutdown_connections 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fdd1b0 0x55893338d110 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.258 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333876b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.256+0000 7f63d73ba740 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f9bd80 0x558933399630 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.257+0000 7f63d73ba740 1 --2- 172.21.15.112:0/2920520587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933387170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.257+0000 7f63adffb640 0 client.16611 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.257+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 >> 172.21.15.112:0/2920520587 conn(0x558932fd1300 msgr2=0x55893337c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.257+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 shutdown_connections 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.257+0000 7f63d73ba740 1 -- 172.21.15.112:0/2920520587 wait complete. 2024-04-25T16:53:10.259 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.257+0000 7f63d73ba740 20 client.16611 trim_cache size 0 max 16384 2024-04-25T16:53:10.275 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 Processor -- start 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 -- start start 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933380ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 0x55893338d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fdd1b0 0x55893339ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63b4011230 con 0x558932fbc4d0 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63b400eb00 con 0x558932fdd1b0 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63d73ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63b400c970 con 0x558932fb42f0 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.273+0000 7f63cd9c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 0x55893338d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 0x55893338d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45752/0 (socket says 172.21.15.112:45752) 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63ce9c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fdd1b0 0x55893339ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 -- 172.21.15.112:0/3974018579 learned_addr learned my addr 172.21.15.112:0/3974018579 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933380ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.276 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 -- 172.21.15.112:0/3974018579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 msgr2=0x558933380ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933380ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 -- 172.21.15.112:0/3974018579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fdd1b0 msgr2=0x55893339ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fdd1b0 0x55893339ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 -- 172.21.15.112:0/3974018579 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558933389b30 con 0x558932fbc4d0 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 0x55893338d010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0a4720 tx=0x7f63bc0e18a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f53fd7f4900499d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63affff640 1 -- 172.21.15.112:0/3974018579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0a7280 con 0x558932fbc4d0 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63affff640 1 -- 172.21.15.112:0/3974018579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc0a6fc0 con 0x558932fbc4d0 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.274+0000 7f63affff640 1 -- 172.21.15.112:0/3974018579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0a6c20 con 0x558932fbc4d0 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 -- 172.21.15.112:0/3974018579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 msgr2=0x55893338d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 0x55893338d010 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0a4720 tx=0x7f63bc0e18a0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 -- 172.21.15.112:0/3974018579 shutdown_connections 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fdd1b0 0x55893339ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.277 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fbc4d0 0x55893338d010 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 --2- 172.21.15.112:0/3974018579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fb42f0 0x558933380ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 -- 172.21.15.112:0/3974018579 >> 172.21.15.112:0/3974018579 conn(0x558932fd1300 msgr2=0x5589333807e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 -- 172.21.15.112:0/3974018579 shutdown_connections 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 -- 172.21.15.112:0/3974018579 wait complete. 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.275+0000 7f63d73ba740 1 -- start start 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 Processor -- start 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 0x5589333932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fdd1b0 0x55893338e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f63b400cfc0 con 0x558932fb42f0 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63b400e430 con 0x558932fbc4d0 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63b400d690 con 0x558932fdd1b0 2024-04-25T16:53:10.278 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63d73ba740 10 client.0 _ll_register_callbacks cb 0x7f63c001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 0x5589333932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63cd9c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 0x5589333932b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:45758/0 (socket says 172.21.15.112:45758) 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 -- 172.21.15.112:0/1561390788 learned_addr learned my addr 172.21.15.112:0/1561390788 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce9c4640 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fdd1b0 0x55893338e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 -- 172.21.15.112:0/1561390788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fdd1b0 msgr2=0x55893338e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fdd1b0 0x55893338e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 -- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 msgr2=0x5589333937f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333937f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 -- 172.21.15.112:0/1561390788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55893338ee50 con 0x558932fb42f0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63ce1c3640 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 0x5589333932b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f63b8032010 tx=0x7f63b80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efde21920172bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.276+0000 7f63adffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63adffb640 1 -- 172.21.15.112:0/1561390788 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8003970 con 0x558932fb42f0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63adffb640 1 -- 172.21.15.112:0/1561390788 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8644+0+0 (secure 0 0 0) 0x7f63b8003b10 con 0x558932fb42f0 2024-04-25T16:53:10.279 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63bc0a6fc0 con 0x558932fb42f0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63adffb640 1 -- 172.21.15.112:0/1561390788 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63b8003d10 con 0x558932fb42f0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 10 client.16617 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589334013f0 con 0x558932fb42f0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63adffb640 1 -- 172.21.15.112:0/1561390788 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc0a6fc0 con 0x558932fb42f0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63adffb640 1 -- 172.21.15.112:0/1561390788 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f63b80042a0 con 0x558932fb42f0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 10 client.16617 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 20 client.16617 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f711c0 0x558933403b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 4 client.16617 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589332eaee0 con 0x558932f711c0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.277+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f711c0 0x558933403b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.278+0000 7f63adffb640 10 client.16617 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:10.280 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.278+0000 7f63cd9c2640 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f711c0 0x558933403b60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2700f918c5f4c7e server_cookie=7ed4b67982deb5f6 in_seq=0 out_seq=0 2024-04-25T16:53:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.794+0000 7f63adffb640 1 -- 172.21.15.112:0/1561390788 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589332eaee0 con 0x558932f711c0 2024-04-25T16:53:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.794+0000 7f63adffb640 10 client.16617 handle_command_reply: tid=0 2024-04-25T16:53:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.794+0000 7f63d73ba740 1 client.16617 shutdown 2024-04-25T16:53:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.794+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 msgr2=0x5589333932b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.794+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 0x5589333932b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f63b8032010 tx=0x7f63b80095c0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.793 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 shutdown_connections 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x558932f711c0 0x558933403b60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63adffb640 0 client.16617 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558932fdd1b0 0x55893338e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558932fbc4d0 0x5589333937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 --2- 172.21.15.112:0/1561390788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x558932fb42f0 0x5589333932b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 >> 172.21.15.112:0/1561390788 conn(0x558932fd1300 msgr2=0x558933385660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 shutdown_connections 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 1 -- 172.21.15.112:0/1561390788 wait complete. 2024-04-25T16:53:10.794 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.795+0000 7f63d73ba740 20 client.16617 trim_cache size 0 max 16384 2024-04-25T16:53:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:53:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:53:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:53:10.795 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:53:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f63b00afc50 msgr2=0x7f63b00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f63b00afc50 0x7f63b00b2160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63b8000f80 tx=0x7f63b8009360 comp rx=0 tx=0).stop 2024-04-25T16:53:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 msgr2=0x7f63c819ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c819ace0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00e9c0 tx=0x7f63bc00ee80 comp rx=0 tx=0).stop 2024-04-25T16:53:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 shutdown_connections 2024-04-25T16:53:10.796 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f63b00afc50 0x7f63b00b2160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63c810e5a0 0x7f63c819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f63c8109bf0 0x7f63c819ace0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 --2- 172.21.15.112:0/1754634080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63c81020a0 0x7f63c819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 >> 172.21.15.112:0/1754634080 conn(0x7f63c80778b0 msgr2=0x7f63c810a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 shutdown_connections 2024-04-25T16:53:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:10.796+0000 7f63d0427640 1 -- 172.21.15.112:0/1754634080 wait complete. 2024-04-25T16:53:11.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:10 smithi112 ceph-mon[29122]: pgmap v470: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 288 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-04-25T16:53:11.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:10 smithi133 ceph-mon[32691]: pgmap v470: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 288 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-04-25T16:53:11.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 22602 18.28 MB/sec warmup 76 sec latency 475.651 ms 2024-04-25T16:53:11.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:10 smithi116 ceph-mon[32712]: pgmap v470: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 288 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-04-25T16:53:12.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 23075 18.52 MB/sec warmup 77 sec latency 217.991 ms 2024-04-25T16:53:13.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:12 smithi112 ceph-mon[29122]: pgmap v471: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 288 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-04-25T16:53:13.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:12 smithi133 ceph-mon[32691]: pgmap v471: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 288 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-04-25T16:53:13.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 23323 18.49 MB/sec warmup 78 sec latency 186.453 ms 2024-04-25T16:53:13.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:12 smithi116 ceph-mon[32712]: pgmap v471: 97 pgs: 27 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 58 active+clean; 288 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 445 op/s 2024-04-25T16:53:14.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 23537 18.38 MB/sec warmup 79 sec latency 336.081 ms 2024-04-25T16:53:15.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:14 smithi112 ceph-mon[29122]: pgmap v472: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 291 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 535 op/s 2024-04-25T16:53:15.198 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:14 smithi133 ceph-mon[32691]: pgmap v472: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 291 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 535 op/s 2024-04-25T16:53:15.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 23728 18.24 MB/sec warmup 80 sec latency 389.719 ms 2024-04-25T16:53:15.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:14 smithi116 ceph-mon[32712]: pgmap v472: 97 pgs: 22 active+clean+snaptrim_wait, 10 active+clean+snaptrim, 65 active+clean; 291 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 535 op/s 2024-04-25T16:53:16.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 23892 18.07 MB/sec warmup 81 sec latency 125.640 ms 2024-04-25T16:53:17.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:16 smithi112 ceph-mon[29122]: pgmap v473: 97 pgs: 97 active+clean; 281 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 675 op/s 2024-04-25T16:53:17.201 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:16 smithi133 ceph-mon[32691]: pgmap v473: 97 pgs: 97 active+clean; 281 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 675 op/s 2024-04-25T16:53:17.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 24196 17.98 MB/sec warmup 82 sec latency 110.467 ms 2024-04-25T16:53:17.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:16 smithi116 ceph-mon[32712]: pgmap v473: 97 pgs: 97 active+clean; 281 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 675 op/s 2024-04-25T16:53:18.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 24607 18.12 MB/sec warmup 83 sec latency 156.547 ms 2024-04-25T16:53:18.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:18 smithi133 ceph-mon[32691]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T16:53:18.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:18 smithi116 ceph-mon[32712]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T16:53:18.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:18 smithi112 ceph-mon[29122]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T16:53:19.204 INFO:tasks.workunit.client.0.smithi112.stdout: 10 25087 18.04 MB/sec warmup 84 sec latency 123.923 ms 2024-04-25T16:53:19.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:19 smithi133 ceph-mon[32691]: pgmap v475: 97 pgs: 97 active+clean; 281 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 675 op/s 2024-04-25T16:53:19.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:19 smithi116 ceph-mon[32712]: pgmap v475: 97 pgs: 97 active+clean; 281 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 675 op/s 2024-04-25T16:53:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:19 smithi112 ceph-mon[29122]: pgmap v475: 97 pgs: 97 active+clean; 281 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.8 MiB/s wr, 675 op/s 2024-04-25T16:53:20.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 25461 18.02 MB/sec warmup 85 sec latency 177.776 ms 2024-04-25T16:53:21.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 25866 18.18 MB/sec warmup 86 sec latency 238.007 ms 2024-04-25T16:53:21.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:21 smithi133 ceph-mon[32691]: pgmap v476: 97 pgs: 97 active+clean; 297 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-04-25T16:53:21.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:21 smithi116 ceph-mon[32712]: pgmap v476: 97 pgs: 97 active+clean; 297 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-04-25T16:53:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:21 smithi112 ceph-mon[29122]: pgmap v476: 97 pgs: 97 active+clean; 297 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-04-25T16:53:22.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 26325 18.41 MB/sec warmup 87 sec latency 204.087 ms 2024-04-25T16:53:23.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 26641 18.40 MB/sec warmup 88 sec latency 234.150 ms 2024-04-25T16:53:23.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:23 smithi133 ceph-mon[32691]: pgmap v477: 97 pgs: 97 active+clean; 297 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-04-25T16:53:23.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:23 smithi116 ceph-mon[32712]: pgmap v477: 97 pgs: 97 active+clean; 297 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-04-25T16:53:23.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:23 smithi112 ceph-mon[29122]: pgmap v477: 97 pgs: 97 active+clean; 297 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.7 MiB/s wr, 498 op/s 2024-04-25T16:53:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 -- 172.21.15.112:0/3584133488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 msgr2=0x7f9970102860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 --2- 172.21.15.112:0/3584133488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f9970102860 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f996c009900 tx=0x7f996c02a410 comp rx=0 tx=0).stop 2024-04-25T16:53:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 -- 172.21.15.112:0/3584133488 shutdown_connections 2024-04-25T16:53:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 --2- 172.21.15.112:0/3584133488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99701074a0 0x7f997010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 --2- 172.21.15.112:0/3584133488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99701069e0 0x7f9970106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 --2- 172.21.15.112:0/3584133488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f9970102860 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 -- 172.21.15.112:0/3584133488 >> 172.21.15.112:0/3584133488 conn(0x7f99700fd8b0 msgr2=0x7f99700ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 -- 172.21.15.112:0/3584133488 shutdown_connections 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 -- 172.21.15.112:0/3584133488 wait complete. 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 Processor -- start 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.899+0000 7f997833f640 1 -- start start 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f997019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99701069e0 0x7f997019ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99701074a0 0x7f997019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9970112c30 con 0x7f9970102450 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9970112ab0 con 0x7f99701069e0 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9970112db0 con 0x7f99701074a0 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f997019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f997019a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36146/0 (socket says 172.21.15.112:36146) 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 -- 172.21.15.112:0/3604743961 learned_addr learned my addr 172.21.15.112:0/3604743961 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:23.900 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99758da640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99701069e0 0x7f997019ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99768dc640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99701074a0 0x7f997019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 -- 172.21.15.112:0/3604743961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99701074a0 msgr2=0x7f997019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99701074a0 0x7f997019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 -- 172.21.15.112:0/3604743961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99701069e0 msgr2=0x7f997019ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99701069e0 0x7f997019ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 -- 172.21.15.112:0/3604743961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997019f640 con 0x7f9970102450 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99760db640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f997019a7e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f996c005940 tx=0x7f996c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99677fe640 1 -- 172.21.15.112:0/3604743961 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c03f030 con 0x7f9970102450 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f99677fe640 1 -- 172.21.15.112:0/3604743961 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996c0044c0 con 0x7f9970102450 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.900+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997019f8d0 con 0x7f9970102450 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.901+0000 7f99677fe640 1 -- 172.21.15.112:0/3604743961 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996c035070 con 0x7f9970102450 2024-04-25T16:53:23.901 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.901+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f997019fe10 con 0x7f9970102450 2024-04-25T16:53:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.903+0000 7f997f1f4740 1 Processor -- start 2024-04-25T16:53:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 -- start start 2024-04-25T16:53:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99677fe640 1 -- 172.21.15.112:0/3604743961 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f996c042030 con 0x7f9970102450 2024-04-25T16:53:23.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 0x55ac1d1d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1d8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1d8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d13e460 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d0f8590 con 0x55ac1ce381b0 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f997f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d13a420 con 0x55ac1ce174d0 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99760db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1d8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 0x55ac1d1d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 0x55ac1d1d7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:49128/0 (socket says 172.21.15.112:49128) 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 -- 172.21.15.112:0/86246686 learned_addr learned my addr 172.21.15.112:0/86246686 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99758da640 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1d8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 -- 172.21.15.112:0/86246686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 msgr2=0x55ac1d1d8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1d8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 -- 172.21.15.112:0/86246686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 msgr2=0x55ac1d1d8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.908 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1d8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99768dc640 1 -- 172.21.15.112:0/86246686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1d1050d0 con 0x55ac1ce381b0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99758da640 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.904+0000 7f99677fe640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f995c0afcd0 0x7f995c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.905+0000 7f99768dc640 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 0x55ac1d1d7af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9968009f50 tx=0x7f99680118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d352315a251cc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.905+0000 7f99760db640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f995c0afcd0 0x7f995c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.905+0000 7f9964bfe640 1 -- 172.21.15.112:0/86246686 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968004d20 con 0x55ac1ce381b0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.905+0000 7f9964bfe640 1 -- 172.21.15.112:0/86246686 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9968004050 con 0x55ac1ce381b0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.905+0000 7f9964bfe640 1 -- 172.21.15.112:0/86246686 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968005280 con 0x55ac1ce381b0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.905+0000 7f99677fe640 1 -- 172.21.15.112:0/3604743961 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f996c0ff370 con 0x7f9970102450 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 -- 172.21.15.112:0/86246686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 msgr2=0x55ac1d1d7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f99760db640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f995c0afcd0 0x7f995c0b21e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f996c0fcb20 tx=0x7f996c102ec0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 0x55ac1d1d7af0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9968009f50 tx=0x7f99680118d0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 -- 172.21.15.112:0/86246686 shutdown_connections 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1d8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1d8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.909 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 --2- 172.21.15.112:0/86246686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce381b0 0x55ac1d1d7af0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 -- 172.21.15.112:0/86246686 >> 172.21.15.112:0/86246686 conn(0x55ac1ce2c300 msgr2=0x55ac1d1d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 -- 172.21.15.112:0/86246686 shutdown_connections 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 -- 172.21.15.112:0/86246686 wait complete. 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 -- start start 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.906+0000 7f997f1f4740 1 Processor -- start 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce381b0 0x55ac1d1e2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d1e4c20 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d0c5b60 con 0x55ac1ce174d0 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ac1d1e4da0 con 0x55ac1ce381b0 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.910 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 10 client.0 _ll_register_callbacks cb 0x55ac1d0ea2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36168/0 (socket says 172.21.15.112:36168) 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 -- 172.21.15.112:0/2593469921 learned_addr learned my addr 172.21.15.112:0/2593469921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99758da640 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 -- 172.21.15.112:0/2593469921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce381b0 msgr2=0x55ac1d1e2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce381b0 0x55ac1d1e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 -- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 msgr2=0x55ac1d1e4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1e4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 -- 172.21.15.112:0/2593469921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1ce50210 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f99768dc640 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e3b30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9968009250 tx=0x7f9968004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985e783662d50b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f9959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f9959ffb640 1 -- 172.21.15.112:0/2593469921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968028070 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f9959ffb640 1 -- 172.21.15.112:0/2593469921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f9968004c00 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9968004050 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f9959ffb640 1 -- 172.21.15.112:0/2593469921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9968004e00 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.911 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.907+0000 7f997f1f4740 10 client.16629 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac1cf1c620 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f9959ffb640 1 -- 172.21.15.112:0/2593469921 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9968004050 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f9959ffb640 1 -- 172.21.15.112:0/2593469921 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9968024570 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 10 client.16629 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 20 client.16629 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdf6d80 0x55ac1d1f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 4 client.16629 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1d14c070 con 0x55ac1cdf6d80 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f99758da640 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdf6d80 0x55ac1d1f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.908+0000 7f9959ffb640 10 client.16629 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:23.912 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.909+0000 7f99758da640 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdf6d80 0x55ac1d1f4670 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa6798b265194e16 server_cookie=97c59328796caa6e in_seq=0 out_seq=0 2024-04-25T16:53:23.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f9959ffb640 1 -- 172.21.15.112:0/2593469921 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac1d14c070 con 0x55ac1cdf6d80 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f9959ffb640 10 client.16629 handle_command_reply: tid=0 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 client.16629 shutdown 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 msgr2=0x55ac1d1e3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e3b30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9968009250 tx=0x7f9968004940 comp rx=0 tx=0).stop 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 shutdown_connections 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdf6d80 0x55ac1d1f4670 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce381b0 0x55ac1d1e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1e4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f9959ffb640 0 client.16629 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2593469921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e3b30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 >> 172.21.15.112:0/2593469921 conn(0x55ac1ce2c300 msgr2=0x55ac1d1d7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 shutdown_connections 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 1 -- 172.21.15.112:0/2593469921 wait complete. 2024-04-25T16:53:23.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.913+0000 7f997f1f4740 20 client.16629 trim_cache size 0 max 16384 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.929+0000 7f997f1f4740 1 Processor -- start 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.929+0000 7f997f1f4740 1 -- start start 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d1fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f997f1f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99701b2450 con 0x55ac1ce381b0 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f997f1f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f99701afd20 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f997f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f99701adb90 con 0x55ac1ce174d0 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d1fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d1fa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36176/0 (socket says 172.21.15.112:36176) 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 -- 172.21.15.112:0/2992209435 learned_addr learned my addr 172.21.15.112:0/2992209435 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99758da640 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 -- 172.21.15.112:0/2992209435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 msgr2=0x55ac1d1f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 -- 172.21.15.112:0/2992209435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce0f2f0 msgr2=0x55ac1d1dbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1dbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 -- 172.21.15.112:0/2992209435 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1d1d8c90 con 0x55ac1ce381b0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99758da640 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f99768dc640 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d1fa2b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f99680249c0 tx=0x7f9968009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5b1188c2500091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f995bfff640 1 -- 172.21.15.112:0/2992209435 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968028030 con 0x55ac1ce381b0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f995bfff640 1 -- 172.21.15.112:0/2992209435 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99680057f0 con 0x55ac1ce381b0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.930+0000 7f995bfff640 1 -- 172.21.15.112:0/2992209435 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968005000 con 0x55ac1ce381b0 2024-04-25T16:53:23.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 -- 172.21.15.112:0/2992209435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 msgr2=0x55ac1d1fa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d1fa2b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f99680249c0 tx=0x7f9968009520 comp rx=0 tx=0).stop 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 -- 172.21.15.112:0/2992209435 shutdown_connections 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce174d0 0x55ac1d1f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d1fa2b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 --2- 172.21.15.112:0/2992209435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 -- 172.21.15.112:0/2992209435 >> 172.21.15.112:0/2992209435 conn(0x55ac1ce2c300 msgr2=0x55ac1d1db7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 -- 172.21.15.112:0/2992209435 shutdown_connections 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 -- 172.21.15.112:0/2992209435 wait complete. 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.931+0000 7f997f1f4740 1 -- start start 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 Processor -- start 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d253ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.934 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f99701ae1e0 con 0x55ac1ce381b0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f99701af650 con 0x55ac1ce174d0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f99701ae8b0 con 0x55ac1ce0f2f0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f997f1f4740 10 client.0 _ll_register_callbacks cb 0x7f996c102d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d253ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d253ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36192/0 (socket says 172.21.15.112:36192) 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 -- 172.21.15.112:0/4024628097 learned_addr learned my addr 172.21.15.112:0/4024628097 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99758da640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 -- 172.21.15.112:0/4024628097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce0f2f0 msgr2=0x55ac1d1e9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 -- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 msgr2=0x55ac1d1ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 -- 172.21.15.112:0/4024628097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1d1ddbc0 con 0x55ac1ce381b0 2024-04-25T16:53:23.935 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f99768dc640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d253ec0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9968009730 tx=0x7f9968009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a3449b355139a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.932+0000 7f9959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f9959ffb640 1 -- 172.21.15.112:0/4024628097 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996804c060 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f9959ffb640 1 -- 172.21.15.112:0/4024628097 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f9968024450 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99680057f0 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f9959ffb640 1 -- 172.21.15.112:0/4024628097 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9968028040 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 10 client.16635 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ac1d25c7e0 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f9959ffb640 1 -- 172.21.15.112:0/4024628097 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99680057f0 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f9959ffb640 1 -- 172.21.15.112:0/4024628097 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9968046030 con 0x55ac1ce381b0 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 10 client.16635 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 20 client.16635 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdcc1c0 0x55ac1d25ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:23.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 4 client.16635 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:53:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac1d145ee0 con 0x55ac1cdcc1c0 2024-04-25T16:53:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.933+0000 7f99758da640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdcc1c0 0x55ac1d25ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.934+0000 7f99758da640 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdcc1c0 0x55ac1d25ee80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745052a3b9838ef6 server_cookie=a097b0808b2d7890 in_seq=0 out_seq=0 2024-04-25T16:53:23.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:23.934+0000 7f9959ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:24.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 26821 18.25 MB/sec warmup 89 sec latency 323.918 ms 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.799+0000 7f9959ffb640 1 -- 172.21.15.112:0/4024628097 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac1d145ee0 con 0x55ac1cdcc1c0 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.799+0000 7f9959ffb640 10 client.16635 handle_command_reply: tid=0 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.799+0000 7f997f1f4740 1 client.16635 shutdown 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.799+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 msgr2=0x55ac1d253ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.799+0000 7f997f1f4740 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d253ec0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9968009730 tx=0x7f9968009f80 comp rx=0 tx=0).stop 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 shutdown_connections 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55ac1ce381b0 0x55ac1d253ec0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.799 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ac1ce174d0 0x55ac1d1ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55ac1cdcc1c0 0x55ac1d25ee80 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 --2- 172.21.15.112:0/4024628097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac1ce0f2f0 0x55ac1d1e9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 >> 172.21.15.112:0/4024628097 conn(0x55ac1ce2c300 msgr2=0x55ac1d1e3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f9959ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 shutdown_connections 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 1 -- 172.21.15.112:0/4024628097 wait complete. 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.800+0000 7f997f1f4740 20 client.16635 trim_cache size 0 max 16384 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:53:24.800 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:53:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f995c0afcd0 msgr2=0x7f995c0b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f995c0afcd0 0x7f995c0b21e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f996c0fcb20 tx=0x7f996c102ec0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 msgr2=0x7f997019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:24.801 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f997019a7e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f996c005940 tx=0x7f996c0040c0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 shutdown_connections 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f995c0afcd0 0x7f995c0b21e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99701074a0 0x7f997019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99701069e0 0x7f997019ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 --2- 172.21.15.112:0/3604743961 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9970102450 0x7f997019a7e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 >> 172.21.15.112:0/3604743961 conn(0x7f99700fd8b0 msgr2=0x7f9970108200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 shutdown_connections 2024-04-25T16:53:24.802 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:24.801+0000 7f997833f640 1 -- 172.21.15.112:0/3604743961 wait complete. 2024-04-25T16:53:25.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 26959 18.11 MB/sec warmup 90 sec latency 943.345 ms 2024-04-25T16:53:25.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:25 smithi133 ceph-mon[32691]: pgmap v478: 97 pgs: 97 active+clean; 300 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 411 op/s 2024-04-25T16:53:25.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:25 smithi116 ceph-mon[32712]: pgmap v478: 97 pgs: 97 active+clean; 300 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 411 op/s 2024-04-25T16:53:25.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:25 smithi112 ceph-mon[29122]: pgmap v478: 97 pgs: 97 active+clean; 300 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 411 op/s 2024-04-25T16:53:26.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 27110 17.95 MB/sec warmup 91 sec latency 248.544 ms 2024-04-25T16:53:27.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 27419 17.96 MB/sec warmup 92 sec latency 83.023 ms 2024-04-25T16:53:27.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:27 smithi133 ceph-mon[32691]: pgmap v479: 97 pgs: 97 active+clean; 308 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 333 op/s 2024-04-25T16:53:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:27 smithi116 ceph-mon[32712]: pgmap v479: 97 pgs: 97 active+clean; 308 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 333 op/s 2024-04-25T16:53:27.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:27 smithi112 ceph-mon[29122]: pgmap v479: 97 pgs: 97 active+clean; 308 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 6.3 MiB/s wr, 333 op/s 2024-04-25T16:53:28.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 27830 18.00 MB/sec warmup 93 sec latency 78.527 ms 2024-04-25T16:53:29.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 28288 18.08 MB/sec warmup 94 sec latency 196.179 ms 2024-04-25T16:53:29.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:29 smithi133 ceph-mon[32691]: pgmap v480: 97 pgs: 97 active+clean; 308 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 315 op/s 2024-04-25T16:53:29.508 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:29 smithi116 ceph-mon[32712]: pgmap v480: 97 pgs: 97 active+clean; 308 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 315 op/s 2024-04-25T16:53:29.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:29 smithi112 ceph-mon[29122]: pgmap v480: 97 pgs: 97 active+clean; 308 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 315 op/s 2024-04-25T16:53:30.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 28706 18.02 MB/sec warmup 95 sec latency 267.710 ms 2024-04-25T16:53:31.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 29073 18.09 MB/sec warmup 96 sec latency 192.668 ms 2024-04-25T16:53:31.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:31 smithi133 ceph-mon[32691]: pgmap v481: 97 pgs: 97 active+clean; 294 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.3 MiB/s wr, 620 op/s 2024-04-25T16:53:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:31 smithi116 ceph-mon[32712]: pgmap v481: 97 pgs: 97 active+clean; 294 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.3 MiB/s wr, 620 op/s 2024-04-25T16:53:31.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:31 smithi112 ceph-mon[29122]: pgmap v481: 97 pgs: 97 active+clean; 294 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 8.3 MiB/s wr, 620 op/s 2024-04-25T16:53:32.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 29494 18.21 MB/sec warmup 97 sec latency 194.234 ms 2024-04-25T16:53:33.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 29855 18.32 MB/sec warmup 98 sec latency 206.845 ms 2024-04-25T16:53:33.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:33 smithi133 ceph-mon[32691]: pgmap v482: 97 pgs: 97 active+clean; 294 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 478 op/s 2024-04-25T16:53:33.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:33 smithi116 ceph-mon[32712]: pgmap v482: 97 pgs: 97 active+clean; 294 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 478 op/s 2024-04-25T16:53:33.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:33 smithi112 ceph-mon[29122]: pgmap v482: 97 pgs: 97 active+clean; 294 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 478 op/s 2024-04-25T16:53:34.207 INFO:tasks.workunit.client.0.smithi112.stdout: 10 30073 18.24 MB/sec warmup 99 sec latency 127.841 ms 2024-04-25T16:53:35.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 30326 18.11 MB/sec warmup 100 sec latency 154.038 ms 2024-04-25T16:53:35.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:35 smithi133 ceph-mon[32691]: pgmap v483: 97 pgs: 97 active+clean; 295 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.1 MiB/s wr, 500 op/s 2024-04-25T16:53:35.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:35 smithi116 ceph-mon[32712]: pgmap v483: 97 pgs: 97 active+clean; 295 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.1 MiB/s wr, 500 op/s 2024-04-25T16:53:35.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:35 smithi112 ceph-mon[29122]: pgmap v483: 97 pgs: 97 active+clean; 295 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.1 MiB/s wr, 500 op/s 2024-04-25T16:53:36.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 30549 18.01 MB/sec warmup 101 sec latency 182.994 ms 2024-04-25T16:53:37.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 30882 17.97 MB/sec warmup 102 sec latency 101.686 ms 2024-04-25T16:53:37.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:37 smithi133 ceph-mon[32691]: pgmap v484: 97 pgs: 97 active+clean; 301 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-04-25T16:53:37.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:37 smithi116 ceph-mon[32712]: pgmap v484: 97 pgs: 97 active+clean; 301 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-04-25T16:53:37.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:37 smithi112 ceph-mon[29122]: pgmap v484: 97 pgs: 97 active+clean; 301 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-04-25T16:53:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 -- 172.21.15.112:0/4155543978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c108ef0 msgr2=0x7f675c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 --2- 172.21.15.112:0/4155543978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c108ef0 0x7f675c102b40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6750009900 tx=0x7f675002a410 comp rx=0 tx=0).stop 2024-04-25T16:53:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 -- 172.21.15.112:0/4155543978 shutdown_connections 2024-04-25T16:53:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 --2- 172.21.15.112:0/4155543978 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c1031b0 0x7f675c10f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.903 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 --2- 172.21.15.112:0/4155543978 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c108ef0 0x7f675c102b40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 --2- 172.21.15.112:0/4155543978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f675c108a50 0x7f675c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 -- 172.21.15.112:0/4155543978 >> 172.21.15.112:0/4155543978 conn(0x7f675c0fe260 msgr2=0x7f675c100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 -- 172.21.15.112:0/4155543978 shutdown_connections 2024-04-25T16:53:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.905+0000 7f6762042640 1 -- 172.21.15.112:0/4155543978 wait complete. 2024-04-25T16:53:37.904 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 Processor -- start 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 -- start start 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 0x7f675c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f675c108a50 0x7f675c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c108ef0 0x7f675c19c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f675c112c70 con 0x7f675c1031b0 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f675c112af0 con 0x7f675c108a50 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f6762042640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f675c112df0 con 0x7f675c108ef0 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.906+0000 7f675b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 0x7f675c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.906 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 0x7f675c19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46810/0 (socket says 172.21.15.112:46810) 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f675c108a50 0x7f675c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 -- 172.21.15.112:0/4039651743 learned_addr learned my addr 172.21.15.112:0/4039651743 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675bfff640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c108ef0 0x7f675c19c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 -- 172.21.15.112:0/4039651743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c108ef0 msgr2=0x7f675c19c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c108ef0 0x7f675c19c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 -- 172.21.15.112:0/4039651743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f675c108a50 msgr2=0x7f675c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f675c108a50 0x7f675c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 -- 172.21.15.112:0/4039651743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c1a6940 con 0x7f675c1031b0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f675b7fe640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 0x7f675c19b630 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f674c00e9f0 tx=0x7f674c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f6758ff9640 1 -- 172.21.15.112:0/4039651743 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c00ccc0 con 0x7f675c1031b0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c1a6c30 con 0x7f675c1031b0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f6758ff9640 1 -- 172.21.15.112:0/4039651743 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c0108e0 con 0x7f675c1031b0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f6758ff9640 1 -- 172.21.15.112:0/4039651743 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c018860 con 0x7f675c1031b0 2024-04-25T16:53:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.907+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c1a71a0 con 0x7f675c1031b0 2024-04-25T16:53:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 Processor -- start 2024-04-25T16:53:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6758ff9640 1 -- 172.21.15.112:0/4039651743 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f674c010430 con 0x7f675c1031b0 2024-04-25T16:53:37.913 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 -- start start 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b103af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b104030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad434d0 0x55dc7b104570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc7b06a460 con 0x55dc7ad641b0 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dc7b024590 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f6768fcb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55dc7b066420 con 0x55dc7ad434d0 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f675bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f675bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b103af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46824/0 (socket says 172.21.15.112:46824) 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f675b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b104030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f675bfff640 1 -- 172.21.15.112:0/3780164053 learned_addr learned my addr 172.21.15.112:0/3780164053 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.910+0000 7f675affd640 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad434d0 0x55dc7b104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675bfff640 1 -- 172.21.15.112:0/3780164053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad434d0 msgr2=0x55dc7b104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.914 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675bfff640 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad434d0 0x55dc7b104570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675bfff640 1 -- 172.21.15.112:0/3780164053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 msgr2=0x55dc7b104030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675bfff640 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b104030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675bfff640 1 -- 172.21.15.112:0/3780164053 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7b0310d0 con 0x55dc7ad641b0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675affd640 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad434d0 0x55dc7b104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f6758ff9640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f67440afc50 0x7f67440b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675bfff640 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b103af0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6754009f90 tx=0x7f67540118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6faaa83c9b6dd52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f675b7fe640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f67440afc50 0x7f67440b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f6758ff9640 1 -- 172.21.15.112:0/4039651743 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f674c0df290 con 0x7f675c1031b0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f6741ffb640 1 -- 172.21.15.112:0/3780164053 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754004d20 con 0x55dc7ad641b0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f6741ffb640 1 -- 172.21.15.112:0/3780164053 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6754004050 con 0x55dc7ad641b0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.911+0000 7f6741ffb640 1 -- 172.21.15.112:0/3780164053 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754005260 con 0x55dc7ad641b0 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 -- 172.21.15.112:0/3780164053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 msgr2=0x55dc7b103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b103af0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6754009f90 tx=0x7f67540118d0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.915 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f675b7fe640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f67440afc50 0x7f67440b2160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f674c0dca40 tx=0x7f674c0e2de0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 -- 172.21.15.112:0/3780164053 shutdown_connections 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad434d0 0x55dc7b104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 --2- 172.21.15.112:0/3780164053 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b103af0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 -- 172.21.15.112:0/3780164053 >> 172.21.15.112:0/3780164053 conn(0x55dc7ad58300 msgr2=0x55dc7b1002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 -- 172.21.15.112:0/3780164053 shutdown_connections 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 -- 172.21.15.112:0/3780164053 wait complete. 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.912+0000 7f6768fcb740 1 -- start start 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 Processor -- start 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55dc7b110c20 con 0x55dc7ad641b0 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dc7aff1b60 con 0x55dc7ad434d0 2024-04-25T16:53:37.916 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55dc7b110da0 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f6768fcb740 10 client.0 _ll_register_callbacks cb 0x55dc7b0162b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675affd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b10e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.112:52816/0 (socket says 172.21.15.112:52816) 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b1140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46830/0 (socket says 172.21.15.112:46830) 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675affd640 1 -- 172.21.15.112:0/4092006720 learned_addr learned my addr 172.21.15.112:0/4092006720 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675bfff640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675affd640 1 -- 172.21.15.112:0/4092006720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad3b2f0 msgr2=0x55dc7b10e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.913+0000 7f675affd640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f675affd640 1 -- 172.21.15.112:0/4092006720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 msgr2=0x55dc7b1140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f675affd640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b1140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f675affd640 1 -- 172.21.15.112:0/4092006720 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7ad7c210 con 0x55dc7ad434d0 2024-04-25T16:53:37.917 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f675bfff640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f675affd640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b10e6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f675c06c500 tx=0x7f6750032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7aeae509bb1de4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f673affd640 10 client.0 ms_handle_connect on v2:172.21.15.116:3300/0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f673affd640 1 -- 172.21.15.112:0/4092006720 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750035040 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6754004050 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f673affd640 1 -- 172.21.15.112:0/4092006720 <== mon.2 v2:172.21.15.116:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f6750031760 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f673affd640 1 -- 172.21.15.112:0/4092006720 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6750004040 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f6768fcb740 10 client.26318 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dc7ae48620 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.914+0000 7f673affd640 1 -- 172.21.15.112:0/4092006720 <== mon.2 v2:172.21.15.116:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6754004050 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f673affd640 1 -- 172.21.15.112:0/4092006720 <== mon.2 v2:172.21.15.116:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6750004750 con 0x55dc7ad434d0 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f6768fcb740 10 client.26318 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f6768fcb740 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f6768fcb740 20 client.26318 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f6768fcb740 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7ad22d80 0x55dc7b120700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.918 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f6768fcb740 4 client.26318 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:53:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc7b078070 con 0x55dc7ad22d80 2024-04-25T16:53:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f675bfff640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7ad22d80 0x55dc7b120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f673affd640 10 client.26318 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.915+0000 7f675bfff640 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7ad22d80 0x55dc7b120700 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90f2898b8280850c server_cookie=4d21d0db19a7c605 in_seq=0 out_seq=0 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f673affd640 1 -- 172.21.15.112:0/4092006720 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc7b078070 con 0x55dc7ad22d80 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f673affd640 10 client.26318 handle_command_reply: tid=0 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 client.26318 shutdown 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 msgr2=0x55dc7b10e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b10e6c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f675c06c500 tx=0x7f6750032e90 comp rx=0 tx=0).stop 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 shutdown_connections 2024-04-25T16:53:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad641b0 0x55dc7b1140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b10e6c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7ad22d80 0x55dc7b120700 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 --2- 172.21.15.112:0/4092006720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f673affd640 0 client.26318 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 >> 172.21.15.112:0/4092006720 conn(0x55dc7ad58300 msgr2=0x55dc7b103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 shutdown_connections 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 1 -- 172.21.15.112:0/4092006720 wait complete. 2024-04-25T16:53:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.917+0000 7f6768fcb740 20 client.26318 trim_cache size 0 max 16384 2024-04-25T16:53:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 Processor -- start 2024-04-25T16:53:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 -- start start 2024-04-25T16:53:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b114010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b125f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1b2570 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1afe40 con 0x55dc7ad434d0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f6768fcb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1adcb0 con 0x55dc7ad641b0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46844/0 (socket says 172.21.15.112:46844) 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 -- 172.21.15.112:0/2336741710 learned_addr learned my addr 172.21.15.112:0/2336741710 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675affd640 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 -- 172.21.15.112:0/2336741710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 msgr2=0x55dc7b125f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b125f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 -- 172.21.15.112:0/2336741710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 msgr2=0x55dc7b114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 -- 172.21.15.112:0/2336741710 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc7b11d230 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.933+0000 7f675bfff640 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10dbc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6754014040 tx=0x7f6754009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b60eeb3c43dcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f67417fa640 1 -- 172.21.15.112:0/2336741710 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67540041e0 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f67417fa640 1 -- 172.21.15.112:0/2336741710 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6754004830 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f67417fa640 1 -- 172.21.15.112:0/2336741710 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675400fc30 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 -- 172.21.15.112:0/2336741710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 msgr2=0x55dc7b10dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10dbc0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6754014040 tx=0x7f6754009500 comp rx=0 tx=0).stop 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 -- 172.21.15.112:0/2336741710 shutdown_connections 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b125f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad434d0 0x55dc7b114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2336741710 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b10dbc0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 -- 172.21.15.112:0/2336741710 >> 172.21.15.112:0/2336741710 conn(0x55dc7ad58300 msgr2=0x55dc7b10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 -- 172.21.15.112:0/2336741710 shutdown_connections 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.934+0000 7f6768fcb740 1 -- 172.21.15.112:0/2336741710 wait complete. 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 -- start start 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 Processor -- start 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b11a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 0x55dc7b115d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b116280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1ae300 con 0x55dc7ad434d0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1af770 con 0x55dc7ad3b2f0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f675c1ae9d0 con 0x55dc7ad641b0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f6768fcb740 10 client.0 _ll_register_callbacks cb 0x7f6750009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f675affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 0x55dc7b115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f675bfff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b11a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f675affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 0x55dc7b115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46848/0 (socket says 172.21.15.112:46848) 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.935+0000 7f675affd640 1 -- 172.21.15.112:0/2919427469 learned_addr learned my addr 172.21.15.112:0/2919427469 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675bfff640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675affd640 1 -- 172.21.15.112:0/2919427469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 msgr2=0x55dc7b116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675affd640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675affd640 1 -- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 msgr2=0x55dc7b11a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675affd640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b11a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675affd640 1 -- 172.21.15.112:0/2919427469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc7b1168f0 con 0x55dc7ad434d0 2024-04-25T16:53:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675bfff640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b11a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f675affd640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 0x55dc7b115d40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6750004650 tx=0x7f6750004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d373c02d5e11ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f673affd640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f673affd640 1 -- 172.21.15.112:0/2919427469 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750035070 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f673affd640 1 -- 172.21.15.112:0/2919427469 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f6750003a50 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6754004830 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f673affd640 1 -- 172.21.15.112:0/2919427469 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f675005e050 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f6768fcb740 10 client.16653 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dc7b1888d0 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.936+0000 7f673affd640 1 -- 172.21.15.112:0/2919427469 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6754004830 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f673affd640 1 -- 172.21.15.112:0/2919427469 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6750055070 con 0x55dc7ad434d0 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f6768fcb740 10 client.16653 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f6768fcb740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f6768fcb740 20 client.16653 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7acf81c0 0x55dc7b18b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f6768fcb740 4 client.16653 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:53:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc7b071ee0 con 0x55dc7acf81c0 2024-04-25T16:53:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f675bfff640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7acf81c0 0x55dc7b18b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f673affd640 10 client.16653 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:37.937+0000 7f675bfff640 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7acf81c0 0x55dc7b18b140 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9af9d40a23bb150c server_cookie=aedef7b12535ecba in_seq=0 out_seq=0 2024-04-25T16:53:38.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 31226 18.03 MB/sec warmup 103 sec latency 320.427 ms 2024-04-25T16:53:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f673affd640 1 -- 172.21.15.112:0/2919427469 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc7b071ee0 con 0x55dc7acf81c0 2024-04-25T16:53:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f673affd640 10 client.16653 handle_command_reply: tid=0 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 client.16653 shutdown 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 msgr2=0x55dc7b115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 0x55dc7b115d40 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6750004650 tx=0x7f6750004410 comp rx=0 tx=0).stop 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 shutdown_connections 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x55dc7acf81c0 0x55dc7b18b140 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dc7ad641b0 0x55dc7b116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f673affd640 0 client.16653 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55dc7ad434d0 0x55dc7b115d40 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 --2- 172.21.15.112:0/2919427469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dc7ad3b2f0 0x55dc7b11a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 >> 172.21.15.112:0/2919427469 conn(0x55dc7ad58300 msgr2=0x55dc7b103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 shutdown_connections 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 1 -- 172.21.15.112:0/2919427469 wait complete. 2024-04-25T16:53:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.483+0000 7f6768fcb740 20 client.16653 trim_cache size 0 max 16384 2024-04-25T16:53:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:53:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:53:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:53:38.483 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f67440afc50 msgr2=0x7f67440b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f67440afc50 0x7f67440b2160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f674c0dca40 tx=0x7f674c0e2de0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 msgr2=0x7f675c19b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 0x7f675c19b630 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f674c00e9f0 tx=0x7f674c00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 shutdown_connections 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f67440afc50 0x7f67440b2160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f675c108ef0 0x7f675c19c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f675c108a50 0x7f675c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.484 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 --2- 172.21.15.112:0/4039651743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f675c1031b0 0x7f675c19b630 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 >> 172.21.15.112:0/4039651743 conn(0x7f675c0fe260 msgr2=0x7f675c0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 shutdown_connections 2024-04-25T16:53:38.485 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:38.484+0000 7f6762042640 1 -- 172.21.15.112:0/4039651743 wait complete. 2024-04-25T16:53:39.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 31605 18.03 MB/sec warmup 104 sec latency 418.824 ms 2024-04-25T16:53:39.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:39 smithi133 ceph-mon[32691]: pgmap v485: 97 pgs: 97 active+clean; 301 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 444 op/s 2024-04-25T16:53:39.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:39 smithi116 ceph-mon[32712]: pgmap v485: 97 pgs: 97 active+clean; 301 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 444 op/s 2024-04-25T16:53:39.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:39 smithi112 ceph-mon[29122]: pgmap v485: 97 pgs: 97 active+clean; 301 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 444 op/s 2024-04-25T16:53:40.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 31972 17.97 MB/sec warmup 105 sec latency 306.077 ms 2024-04-25T16:53:41.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 32322 17.99 MB/sec warmup 106 sec latency 183.003 ms 2024-04-25T16:53:41.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:41 smithi133 ceph-mon[32691]: pgmap v486: 97 pgs: 97 active+clean; 290 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 734 op/s 2024-04-25T16:53:41.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:41 smithi116 ceph-mon[32712]: pgmap v486: 97 pgs: 97 active+clean; 290 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 734 op/s 2024-04-25T16:53:41.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:41 smithi112 ceph-mon[29122]: pgmap v486: 97 pgs: 97 active+clean; 290 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 6.0 MiB/s wr, 734 op/s 2024-04-25T16:53:42.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 32781 18.15 MB/sec warmup 107 sec latency 191.840 ms 2024-04-25T16:53:43.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 33150 18.23 MB/sec warmup 108 sec latency 215.954 ms 2024-04-25T16:53:43.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:43 smithi116 ceph-mon[32712]: pgmap v487: 97 pgs: 97 active+clean; 290 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 391 op/s 2024-04-25T16:53:43.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:43 smithi112 ceph-mon[29122]: pgmap v487: 97 pgs: 97 active+clean; 290 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 391 op/s 2024-04-25T16:53:43.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:43 smithi133 ceph-mon[32691]: pgmap v487: 97 pgs: 97 active+clean; 290 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.9 MiB/s wr, 391 op/s 2024-04-25T16:53:44.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 33403 18.18 MB/sec warmup 109 sec latency 209.617 ms 2024-04-25T16:53:45.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 33617 18.10 MB/sec warmup 110 sec latency 193.000 ms 2024-04-25T16:53:45.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:45 smithi116 ceph-mon[32712]: pgmap v488: 97 pgs: 97 active+clean; 297 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-04-25T16:53:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:45 smithi112 ceph-mon[29122]: pgmap v488: 97 pgs: 97 active+clean; 297 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-04-25T16:53:45.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:45 smithi133 ceph-mon[32691]: pgmap v488: 97 pgs: 97 active+clean; 297 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.8 MiB/s wr, 464 op/s 2024-04-25T16:53:46.205 INFO:tasks.workunit.client.0.smithi112.stdout: 10 33878 17.99 MB/sec warmup 111 sec latency 189.148 ms 2024-04-25T16:53:47.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 34176 17.96 MB/sec warmup 112 sec latency 190.231 ms 2024-04-25T16:53:47.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:47 smithi116 ceph-mon[32712]: pgmap v489: 97 pgs: 97 active+clean; 298 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 486 op/s 2024-04-25T16:53:47.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:47 smithi112 ceph-mon[29122]: pgmap v489: 97 pgs: 97 active+clean; 298 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 486 op/s 2024-04-25T16:53:47.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:47 smithi133 ceph-mon[32691]: pgmap v489: 97 pgs: 97 active+clean; 298 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.9 MiB/s wr, 486 op/s 2024-04-25T16:53:48.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 34602 18.03 MB/sec warmup 113 sec latency 189.982 ms 2024-04-25T16:53:49.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 35084 18.09 MB/sec warmup 114 sec latency 195.622 ms 2024-04-25T16:53:49.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:49 smithi116 ceph-mon[32712]: pgmap v490: 97 pgs: 97 active+clean; 298 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 406 op/s 2024-04-25T16:53:49.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:49 smithi112 ceph-mon[29122]: pgmap v490: 97 pgs: 97 active+clean; 298 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 406 op/s 2024-04-25T16:53:49.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:49 smithi133 ceph-mon[32691]: pgmap v490: 97 pgs: 97 active+clean; 298 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 406 op/s 2024-04-25T16:53:50.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 35448 18.05 MB/sec warmup 115 sec latency 303.996 ms 2024-04-25T16:53:50.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:53:50.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:53:50.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:53:51.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 35797 18.09 MB/sec warmup 116 sec latency 264.985 ms 2024-04-25T16:53:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3571039606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810e0d0 msgr2=0x7f8eb81105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3571039606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810e0d0 0x7f8eb81105c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009ca0 tx=0x7f8eb002a8c0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3571039606 shutdown_connections 2024-04-25T16:53:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3571039606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810e0d0 0x7f8eb81105c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3571039606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb810b6e0 0x7f8eb810db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.587 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3571039606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb8104c80 0x7f8eb8105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3571039606 >> 172.21.15.112:0/3571039606 conn(0x7f8eb8078150 msgr2=0x7f8eb8078590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3571039606 shutdown_connections 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.587+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3571039606 wait complete. 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 Processor -- start 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 -- start start 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb8104c80 0x7f8eb8102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 0x7f8eb8102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb810e0d0 0x7f8eb8102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8113d30 con 0x7f8eb810b6e0 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8113bb0 con 0x7f8eb8104c80 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8ebd84a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8113eb0 con 0x7f8eb810e0d0 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 0x7f8eb8102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 0x7f8eb8102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36722/0 (socket says 172.21.15.112:36722) 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8eb67fc640 1 -- 172.21.15.112:0/3332129836 learned_addr learned my addr 172.21.15.112:0/3332129836 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb8104c80 0x7f8eb8102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.588+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb810e0d0 0x7f8eb8102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb67fc640 1 -- 172.21.15.112:0/3332129836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb810e0d0 msgr2=0x7f8eb8102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb810e0d0 0x7f8eb8102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb67fc640 1 -- 172.21.15.112:0/3332129836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb8104c80 msgr2=0x7f8eb8102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb8104c80 0x7f8eb8102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb67fc640 1 -- 172.21.15.112:0/3332129836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb81a8a10 con 0x7f8eb810b6e0 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb8104c80 0x7f8eb8102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 0x7f8eb8102a80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ea60 tx=0x7f8eac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8ebc848640 1 -- 172.21.15.112:0/3332129836 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac00cde0 con 0x7f8eb810b6e0 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb81a8d00 con 0x7f8eb810b6e0 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8ebc848640 1 -- 172.21.15.112:0/3332129836 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8eac0035f0 con 0x7f8eb810b6e0 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8ebc848640 1 -- 172.21.15.112:0/3332129836 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac010690 con 0x7f8eb810b6e0 2024-04-25T16:53:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.589+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a9190 con 0x7f8eb810b6e0 2024-04-25T16:53:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 Processor -- start 2024-04-25T16:53:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ebc848640 1 -- 172.21.15.112:0/3332129836 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f8eac0037a0 con 0x7f8eb810b6e0 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 -- start start 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede19af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede1a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589edd80460 con 0x5589eda512f0 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5589edd3a590 con 0x5589eda7a1b0 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589edd7c420 con 0x5589eda594d0 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36734/0 (socket says 172.21.15.112:36734) 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede19af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/1930502336 learned_addr learned my addr 172.21.15.112:0/1930502336 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8ebc848640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8ea00afc50 0x7f8ea00b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/1930502336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 msgr2=0x5589ede1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede1a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.592+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/1930502336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 msgr2=0x5589ede19af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede19af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/1930502336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589edd470d0 con 0x5589eda512f0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede19af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8ea00afc50 0x7f8ea00b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8ebc848640 1 -- 172.21.15.112:0/3332129836 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f8eac0df2e0 con 0x7f8eb810b6e0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1a030 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8011540 tx=0x7f8ea8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838a17b6300f4f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8e9dffb640 1 -- 172.21.15.112:0/1930502336 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea800f640 con 0x5589eda512f0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8ea00afc50 0x7f8ea00b2160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5589ede16c60 tx=0x7f8eb00095c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8e9dffb640 1 -- 172.21.15.112:0/1930502336 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea800fc90 con 0x5589eda512f0 2024-04-25T16:53:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.593+0000 7f8e9dffb640 1 -- 172.21.15.112:0/1930502336 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8019c80 con 0x5589eda512f0 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 -- 172.21.15.112:0/1930502336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 msgr2=0x5589ede1a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1a030 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8011540 tx=0x7f8ea8011a00 comp rx=0 tx=0).stop 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 -- 172.21.15.112:0/1930502336 shutdown_connections 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1a030 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/1930502336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede19af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 -- 172.21.15.112:0/1930502336 >> 172.21.15.112:0/1930502336 conn(0x5589eda6e300 msgr2=0x5589ede162a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 -- 172.21.15.112:0/1930502336 shutdown_connections 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 -- 172.21.15.112:0/1930502336 wait complete. 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.594+0000 7f8ec47d8740 1 -- start start 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 Processor -- start 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda512f0 0x5589ede241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede24710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.600 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede2a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x5589ede26c20 con 0x5589eda7a1b0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5589edd07b60 con 0x5589eda512f0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5589ede26da0 con 0x5589eda594d0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8ec47d8740 10 client.0 _ll_register_callbacks cb 0x5589edd2c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede2a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede24710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede2a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36742/0 (socket says 172.21.15.112:36742) 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8eb77fe640 1 -- 172.21.15.112:0/552900726 learned_addr learned my addr 172.21.15.112:0/552900726 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.595+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda512f0 0x5589ede241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb67fc640 1 -- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda512f0 msgr2=0x5589ede241b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda512f0 0x5589ede241b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb67fc640 1 -- 172.21.15.112:0/552900726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 msgr2=0x5589ede2a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede2a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.601 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb67fc640 1 -- 172.21.15.112:0/552900726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589eda92210 con 0x5589eda594d0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede2a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede24710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0dc9e0 tx=0x7f8eac003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b9cae3888ac8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8e96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8e96ffd640 1 -- 172.21.15.112:0/552900726 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac0a6f40 con 0x5589eda594d0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea800fc90 con 0x5589eda594d0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8e96ffd640 1 -- 172.21.15.112:0/552900726 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f8eac0a36c0 con 0x5589eda594d0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8e96ffd640 1 -- 172.21.15.112:0/552900726 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8eac0a7b30 con 0x5589eda594d0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8ec47d8740 10 client.26239 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589edb5e620 con 0x5589eda594d0 2024-04-25T16:53:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.596+0000 7f8e96ffd640 1 -- 172.21.15.112:0/552900726 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea800fc90 con 0x5589eda594d0 2024-04-25T16:53:51.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:51 smithi112 ceph-mon[29122]: pgmap v491: 97 pgs: 97 active+clean; 308 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 528 op/s 2024-04-25T16:53:51.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:53:51.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:53:51.603 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8e96ffd640 1 -- 172.21.15.112:0/552900726 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8eac0c82f0 con 0x5589eda594d0 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8ec47d8740 10 client.26239 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8ec47d8740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8ec47d8740 20 client.26239 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda38d80 0x5589ede367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8ec47d8740 4 client.26239 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589edd8e070 con 0x5589eda38d80 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda38d80 0x5589ede367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8e96ffd640 10 client.26239 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.597+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda38d80 0x5589ede367f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913cc06612063001 server_cookie=bff2bd1a32f20b3e in_seq=0 out_seq=0 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8e96ffd640 1 -- 172.21.15.112:0/552900726 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5589edd8e070 con 0x5589eda38d80 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8e96ffd640 10 client.26239 handle_command_reply: tid=0 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 client.26239 shutdown 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 msgr2=0x5589ede24710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede24710 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0dc9e0 tx=0x7f8eac003cf0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 shutdown_connections 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda38d80 0x5589ede367f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede24710 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede2a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/552900726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda512f0 0x5589ede241b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8e96ffd640 0 client.26239 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 >> 172.21.15.112:0/552900726 conn(0x5589eda6e300 msgr2=0x5589ede1b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 shutdown_connections 2024-04-25T16:53:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 1 -- 172.21.15.112:0/552900726 wait complete. 2024-04-25T16:53:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.602+0000 7f8ec47d8740 20 client.26239 trim_cache size 0 max 16384 2024-04-25T16:53:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 Processor -- start 2024-04-25T16:53:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 -- start start 2024-04-25T16:53:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede2a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.619 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b46c0 con 0x5589eda512f0 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b1f90 con 0x5589eda7a1b0 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81afe00 con 0x5589eda594d0 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:36744/0 (socket says 172.21.15.112:36744) 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/881863918 learned_addr learned my addr 172.21.15.112:0/881863918 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede2a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/881863918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 msgr2=0x5589ede2a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede2a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/881863918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 msgr2=0x5589ede24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.619+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede24510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.620+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/881863918 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589ede16e90 con 0x5589eda512f0 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.620+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1beb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8011510 tx=0x7f8ea8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f963a98016a2095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.620+0000 7f8e9d7fa640 1 -- 172.21.15.112:0/881863918 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8005340 con 0x5589eda512f0 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.620+0000 7f8e9d7fa640 1 -- 172.21.15.112:0/881863918 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea80054e0 con 0x5589eda512f0 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.620+0000 7f8e9d7fa640 1 -- 172.21.15.112:0/881863918 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8019950 con 0x5589eda512f0 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 -- 172.21.15.112:0/881863918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 msgr2=0x5589ede1beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1beb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8011510 tx=0x7f8ea8005040 comp rx=0 tx=0).stop 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 -- 172.21.15.112:0/881863918 shutdown_connections 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda594d0 0x5589ede2a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda7a1b0 0x5589ede24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/881863918 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda512f0 0x5589ede1beb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 -- 172.21.15.112:0/881863918 >> 172.21.15.112:0/881863918 conn(0x5589eda6e300 msgr2=0x5589ede1b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 -- 172.21.15.112:0/881863918 shutdown_connections 2024-04-25T16:53:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 -- 172.21.15.112:0/881863918 wait complete. 2024-04-25T16:53:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 -- start start 2024-04-25T16:53:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.621+0000 7f8ec47d8740 1 Processor -- start 2024-04-25T16:53:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 0x5589ede14e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda594d0 0x5589ede15360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b0450 con 0x5589eda7a1b0 2024-04-25T16:53:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b18c0 con 0x5589eda594d0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81b0b20 con 0x5589eda512f0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8ec47d8740 10 client.0 _ll_register_callbacks cb 0x7f8eac00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 0x5589ede14e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb67fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda594d0 0x5589ede15360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 0x5589ede14e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:35702/0 (socket says 172.21.15.112:35702) 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/245967960 learned_addr learned my addr 172.21.15.112:0/245967960 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda594d0 msgr2=0x5589ede15360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda594d0 0x5589ede15360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/245967960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 msgr2=0x5589ede158a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede158a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb6ffd640 1 -- 172.21.15.112:0/245967960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589ede9cd90 con 0x5589eda512f0 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda594d0 0x5589ede15360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.622+0000 7f8eb77fe640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8eb6ffd640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 0x5589ede14e20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800fc40 tx=0x7f8ea80093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27cee78c624c803f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8e96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8e96ffd640 1 -- 172.21.15.112:0/245967960 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8005340 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8e96ffd640 1 -- 172.21.15.112:0/245967960 <== mon.1 v2:172.21.15.133:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f8ea8032070 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea80054e0 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8e96ffd640 1 -- 172.21.15.112:0/245967960 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8044050 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8ec47d8740 10 client.26245 fetch_fsmap learned FSMap version 39 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5589ede9e020 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8e96ffd640 1 -- 172.21.15.112:0/245967960 <== mon.1 v2:172.21.15.133:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ea80054e0 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8e96ffd640 1 -- 172.21.15.112:0/245967960 <== mon.1 v2:172.21.15.133:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f8ea803f030 con 0x5589eda512f0 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8ec47d8740 10 client.26245 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8ec47d8740 10 client.26245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T16:53:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.623+0000 7f8ec47d8740 20 client.26245 populate_metadata read hostname 'smithi112' 2024-04-25T16:53:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.624+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda0e1c0 0x5589edea05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:53:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.624+0000 7f8ec47d8740 4 client.26245 mds_command: new command op to 24560 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T16:53:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.624+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 --> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589edd87ee0 con 0x5589eda0e1c0 2024-04-25T16:53:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.624+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda0e1c0 0x5589edea05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:53:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.624+0000 7f8e96ffd640 10 client.26245 ms_handle_connect on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:51.624+0000 7f8eb67fc640 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda0e1c0 0x5589edea05b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a477e9b2e9e132 server_cookie=3da9caeeb5edb8a3 in_seq=0 out_seq=0 2024-04-25T16:53:51.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:51 smithi133 ceph-mon[32691]: pgmap v491: 97 pgs: 97 active+clean; 308 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 528 op/s 2024-04-25T16:53:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:53:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:53:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:53:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:51 smithi116 ceph-mon[32712]: pgmap v491: 97 pgs: 97 active+clean; 308 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 5.5 MiB/s wr, 528 op/s 2024-04-25T16:53:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:53:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:53:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:53:52.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 36020 18.12 MB/sec warmup 117 sec latency 361.236 ms 2024-04-25T16:53:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.272+0000 7f8e96ffd640 1 -- 172.21.15.112:0/245967960 <== mds.0 v2:172.21.15.116:6838/1016899540 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589edd87ee0 con 0x5589eda0e1c0 2024-04-25T16:53:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8e96ffd640 10 client.26245 handle_command_reply: tid=0 2024-04-25T16:53:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 client.26245 shutdown 2024-04-25T16:53:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 msgr2=0x5589ede14e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 0x5589ede14e20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800fc40 tx=0x7f8ea80093a0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 shutdown_connections 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x5589eda7a1b0 0x5589ede158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5589eda594d0 0x5589ede15360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.116:6838/1016899540,v1:172.21.15.116:6839/1016899540] conn(0x5589eda0e1c0 0x5589edea05b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 --2- 172.21.15.112:0/245967960 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5589eda512f0 0x5589ede14e20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 >> 172.21.15.112:0/245967960 conn(0x5589eda6e300 msgr2=0x5589ede1a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8e96ffd640 0 client.26245 ms_handle_reset on v2:172.21.15.116:6838/1016899540 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 shutdown_connections 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 1 -- 172.21.15.112:0/245967960 wait complete. 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.273+0000 7f8ec47d8740 20 client.26245 trim_cache size 0 max 16384 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:{ 2024-04-25T16:53:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "message": "", 2024-04-25T16:53:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout: "return_code": 0 2024-04-25T16:53:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stdout:} 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8ea00afc50 msgr2=0x7f8ea00b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8ea00afc50 0x7f8ea00b2160 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5589ede16c60 tx=0x7f8eb00095c0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 msgr2=0x7f8eb8102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 0x7f8eb8102a80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ea60 tx=0x7f8eac00ef20 comp rx=0 tx=0).stop 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 shutdown_connections 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8ea00afc50 0x7f8ea00b2160 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb810e0d0 0x7f8eb8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8eb810b6e0 0x7f8eb8102a80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 --2- 172.21.15.112:0/3332129836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8eb8104c80 0x7f8eb8102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.274+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 >> 172.21.15.112:0/3332129836 conn(0x7f8eb8078150 msgr2=0x7f8eb8101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.275+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 shutdown_connections 2024-04-25T16:53:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi112.front.sepia.ceph.com.smithi112.stderr:2024-04-25T16:53:52.275+0000 7f8ebd84a640 1 -- 172.21.15.112:0/3332129836 wait complete. 2024-04-25T16:53:53.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 36395 18.18 MB/sec warmup 118 sec latency 124.781 ms 2024-04-25T16:53:53.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:53 smithi112 ceph-mon[29122]: pgmap v492: 97 pgs: 97 active+clean; 308 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 238 op/s 2024-04-25T16:53:53.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:53 smithi133 ceph-mon[32691]: pgmap v492: 97 pgs: 97 active+clean; 308 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 238 op/s 2024-04-25T16:53:53.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:53 smithi116 ceph-mon[32712]: pgmap v492: 97 pgs: 97 active+clean; 308 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.2 MiB/s wr, 238 op/s 2024-04-25T16:53:54.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 36631 18.13 MB/sec warmup 119 sec latency 169.415 ms 2024-04-25T16:53:55.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:55 smithi112 ceph-mon[29122]: pgmap v493: 97 pgs: 97 active+clean; 289 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 367 op/s 2024-04-25T16:53:55.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:55 smithi133 ceph-mon[32691]: pgmap v493: 97 pgs: 97 active+clean; 289 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 367 op/s 2024-04-25T16:53:55.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:55 smithi116 ceph-mon[32712]: pgmap v493: 97 pgs: 97 active+clean; 289 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.8 MiB/s wr, 367 op/s 2024-04-25T16:53:56.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 37013 5.84 MB/sec execute 1 sec latency 247.456 ms 2024-04-25T16:53:57.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 37289 6.19 MB/sec execute 2 sec latency 174.616 ms 2024-04-25T16:53:57.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:57 smithi112 ceph-mon[29122]: pgmap v494: 97 pgs: 97 active+clean; 295 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 498 op/s 2024-04-25T16:53:57.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:57 smithi133 ceph-mon[32691]: pgmap v494: 97 pgs: 97 active+clean; 295 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 498 op/s 2024-04-25T16:53:57.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:57 smithi116 ceph-mon[32712]: pgmap v494: 97 pgs: 97 active+clean; 295 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.3 MiB/s wr, 498 op/s 2024-04-25T16:53:58.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 37616 8.98 MB/sec execute 3 sec latency 227.471 ms 2024-04-25T16:53:59.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 38028 12.98 MB/sec execute 4 sec latency 226.937 ms 2024-04-25T16:53:59.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:53:59 smithi112 ceph-mon[29122]: pgmap v495: 97 pgs: 97 active+clean; 295 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-04-25T16:53:59.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:53:59 smithi133 ceph-mon[32691]: pgmap v495: 97 pgs: 97 active+clean; 295 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-04-25T16:53:59.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:53:59 smithi116 ceph-mon[32712]: pgmap v495: 97 pgs: 97 active+clean; 295 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.9 MiB/s wr, 454 op/s 2024-04-25T16:54:00.206 INFO:tasks.workunit.client.0.smithi112.stdout: 10 38429 15.85 MB/sec execute 5 sec latency 249.671 ms 2024-04-25T16:54:01.065 INFO:tasks.workunit.client.0.smithi112.stdout:[41162] write failed on handle 10247 (Resource temporarily unavailable) 2024-04-25T16:54:01.065 INFO:tasks.workunit.client.0.smithi112.stdout:Child failed with status 1 2024-04-25T16:54:01.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:54:01.077 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-25T16:54:01.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T16:54:01.360 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:01 smithi112 ceph-mon[29122]: pgmap v496: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2024-04-25T16:54:01.361 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-25T16:54:01.670 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=551663228469486bbbc00d4e86b7a1a9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi112 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-25T16:54:01.673 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T16:54:01.684 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:54:01.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:01 smithi133 ceph-mon[32691]: pgmap v496: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2024-04-25T16:54:01.699 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T16:54:01.709 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T16:54:01.709 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T16:54:01.709 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:54:01.709 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T16:54:01.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:01 smithi116 ceph-mon[32712]: pgmap v496: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 7.6 MiB/s wr, 588 op/s 2024-04-25T16:54:01.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T16:54:01.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T16:54:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/1444819029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 msgr2=0x7fa7a81105e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:01.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/1444819029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a81105e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0009ca0 tx=0x7fa7a002a950 comp rx=0 tx=0).stop 2024-04-25T16:54:01.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/1444819029 shutdown_connections 2024-04-25T16:54:01.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/1444819029 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a81105e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:01.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/1444819029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7a810b790 0x7fa7a810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:01.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/1444819029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7a8104d30 0x7fa7a8105140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:01.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/1444819029 >> 172.21.15.112:0/1444819029 conn(0x7fa7a80fe260 msgr2=0x7fa7a8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/1444819029 shutdown_connections 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.984+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/1444819029 wait complete. 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 Processor -- start 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 -- start start 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7a8104d30 0x7fa7a819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7a810b790 0x7fa7a819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:01.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8113d50 con 0x7fa7a810e0f0 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8113bd0 con 0x7fa7a810b790 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.985+0000 7fa7ae3b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7a8113ed0 con 0x7fa7a8104d30 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59740/0 (socket says 172.21.15.112:59740) 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 -- 172.21.15.112:0/2406496282 learned_addr learned my addr 172.21.15.112:0/2406496282 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7a77fe640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7a810b790 0x7fa7a819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7a7fff640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7a8104d30 0x7fa7a819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:01.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 -- 172.21.15.112:0/2406496282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7a8104d30 msgr2=0x7fa7a819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7a8104d30 0x7fa7a819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 -- 172.21.15.112:0/2406496282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7a810b790 msgr2=0x7fa7a819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7a810b790 0x7fa7a819ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 -- 172.21.15.112:0/2406496282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7a819f510 con 0x7fa7a810e0f0 2024-04-25T16:54:01.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ac94f640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a819eed0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002ae70 tx=0x7fa7a002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0034040 con 0x7fa7a810e0f0 2024-04-25T16:54:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7a00308f0 con 0x7fa7a810e0f0 2024-04-25T16:54:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7a819f7a0 con 0x7fa7a810e0f0 2024-04-25T16:54:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7a00047f0 con 0x7fa7a810e0f0 2024-04-25T16:54:01.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.986+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7a819fc90 con 0x7fa7a810e0f0 2024-04-25T16:54:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.987+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7a8050740 con 0x7fa7a810e0f0 2024-04-25T16:54:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.989+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fa7a0037020 con 0x7fa7a810e0f0 2024-04-25T16:54:01.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.989+0000 7fa7a57fa640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa7900afc50 0x7fa7900b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.989+0000 7fa7a7fff640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa7900afc50 0x7fa7900b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.989+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7fa7a01000c0 con 0x7fa7a810e0f0 2024-04-25T16:54:01.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.990+0000 7fa7a7fff640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa7900afc50 0x7fa7900b2160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa798005fd0 tx=0x7fa79800b390 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:01.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:01.993+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7a0080ff0 con 0x7fa7a810e0f0 2024-04-25T16:54:02.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.193+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa7a810ec10 con 0x7fa7900afc50 2024-04-25T16:54:02.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.202+0000 7fa7a57fa640 1 -- 172.21.15.112:0/2406496282 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fa7a810ec10 con 0x7fa7900afc50 2024-04-25T16:54:02.200 INFO:teuthology.orchestra.run.smithi112.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T16:38:29", "first": "2024-04-25T16:39:00", "last": "2024-04-25T16:54:00", "last_pruned": "2024-04-25T16:54:01", "created_count": 16, "pruned_count": 10, "active": true} 2024-04-25T16:54:02.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa7900afc50 msgr2=0x7fa7900b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa7900afc50 0x7fa7900b2160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa798005fd0 tx=0x7fa79800b390 comp rx=0 tx=0).stop 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 msgr2=0x7fa7a819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a819eed0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa7a002ae70 tx=0x7fa7a002f5d0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 shutdown_connections 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa7900afc50 0x7fa7900b2160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7a810e0f0 0x7fa7a819eed0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7a810b790 0x7fa7a819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 --2- 172.21.15.112:0/2406496282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7a8104d30 0x7fa7a819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.204+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 >> 172.21.15.112:0/2406496282 conn(0x7fa7a80fe260 msgr2=0x7fa7a810e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.205+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 shutdown_connections 2024-04-25T16:54:02.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.205+0000 7fa7ae3b2640 1 -- 172.21.15.112:0/2406496282 wait complete. 2024-04-25T16:54:02.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T16:54:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 -- 172.21.15.112:0/1201242771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff724109c00 msgr2=0x7ff724101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1201242771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff724109c00 0x7ff724101b70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff72002a410 comp rx=0 tx=0).stop 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 -- 172.21.15.112:0/1201242771 shutdown_connections 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1201242771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff72410e5b0 0x7ff7241109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1201242771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7241020b0 0x7ff7241024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1201242771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff724109c00 0x7ff724101b70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 -- 172.21.15.112:0/1201242771 >> 172.21.15.112:0/1201242771 conn(0x7ff7240fd8a0 msgr2=0x7ff7240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.327+0000 7ff72c2af640 1 -- 172.21.15.112:0/1201242771 shutdown_connections 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 -- 172.21.15.112:0/1201242771 wait complete. 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 Processor -- start 2024-04-25T16:54:02.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 -- start start 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7241020b0 0x7ff72419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff724109c00 0x7ff72419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 0x7ff72419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff724113d70 con 0x7ff72410e5b0 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff724113bf0 con 0x7ff7241020b0 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.328+0000 7ff72c2af640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff724113ef0 con 0x7ff724109c00 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 0x7ff72419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 0x7ff72419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59746/0 (socket says 172.21.15.112:59746) 2024-04-25T16:54:02.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 -- 172.21.15.112:0/1464218102 learned_addr learned my addr 172.21.15.112:0/1464218102 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a04b640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7241020b0 0x7ff72419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72984a640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff724109c00 0x7ff72419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 -- 172.21.15.112:0/1464218102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff724109c00 msgr2=0x7ff72419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff724109c00 0x7ff72419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 -- 172.21.15.112:0/1464218102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7241020b0 msgr2=0x7ff72419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7241020b0 0x7ff72419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 -- 172.21.15.112:0/1464218102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72419f5b0 con 0x7ff72410e5b0 2024-04-25T16:54:02.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72a84c640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 0x7ff72419ef70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00eb40 tx=0x7ff71c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c004b20 con 0x7ff72410e5b0 2024-04-25T16:54:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff71c004cc0 con 0x7ff72410e5b0 2024-04-25T16:54:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72419f8a0 con 0x7ff72410e5b0 2024-04-25T16:54:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c010720 con 0x7ff72410e5b0 2024-04-25T16:54:02.331 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.329+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241a7220 con 0x7ff72410e5b0 2024-04-25T16:54:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.330+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff724050740 con 0x7ff72410e5b0 2024-04-25T16:54:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.332+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7ff71c005560 con 0x7ff72410e5b0 2024-04-25T16:54:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.332+0000 7ff71b7fe640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff7100afc50 0x7ff7100b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.332+0000 7ff72a04b640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff7100afc50 0x7ff7100b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.332+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7ff71c0df810 con 0x7ff72410e5b0 2024-04-25T16:54:02.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.333+0000 7ff72a04b640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff7100afc50 0x7ff7100b2160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff720009230 tx=0x7ff720031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:02.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.336+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff71c0a8060 con 0x7ff72410e5b0 2024-04-25T16:54:02.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.535+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 --> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7ff7240fe3c0 con 0x7ff7100afc50 2024-04-25T16:54:02.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.545+0000 7ff71b7fe640 1 -- 172.21.15.112:0/1464218102 <== mgr.15294 v2:172.21.15.112:6800/779201591 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7ff7240fe3c0 con 0x7ff7100afc50 2024-04-25T16:54:02.543 INFO:teuthology.orchestra.run.smithi112.stdout:/ 1m 6m3h 2024-04-25T16:54:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff7100afc50 msgr2=0x7ff7100b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff7100afc50 0x7ff7100b2160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff720009230 tx=0x7ff720031400 comp rx=0 tx=0).stop 2024-04-25T16:54:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 msgr2=0x7ff72419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 0x7ff72419ef70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00eb40 tx=0x7ff71c00c2d0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 shutdown_connections 2024-04-25T16:54:02.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7ff7100afc50 0x7ff7100b2160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff724109c00 0x7ff72419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff72410e5b0 0x7ff72419ef70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 --2- 172.21.15.112:0/1464218102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7241020b0 0x7ff72419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 >> 172.21.15.112:0/1464218102 conn(0x7ff7240fd8a0 msgr2=0x7ff7240fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:02.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 shutdown_connections 2024-04-25T16:54:02.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.547+0000 7ff72c2af640 1 -- 172.21.15.112:0/1464218102 wait complete. 2024-04-25T16:54:02.557 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T16:54:02.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:54:02.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 -- 172.21.15.112:0/2683393232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 msgr2=0x7f4fa8103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/2683393232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa8103c10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c009900 tx=0x7f4f9c02a410 comp rx=0 tx=0).stop 2024-04-25T16:54:02.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 -- 172.21.15.112:0/2683393232 shutdown_connections 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/2683393232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fa8104280 0x7f4fa8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/2683393232 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa8103c10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/2683393232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 0x7f4fa81036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 -- 172.21.15.112:0/2683393232 >> 172.21.15.112:0/2683393232 conn(0x7f4fa80ff210 msgr2=0x7f4fa8101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 -- 172.21.15.112:0/2683393232 shutdown_connections 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.679+0000 7f4fb03ab640 1 -- 172.21.15.112:0/2683393232 wait complete. 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 Processor -- start 2024-04-25T16:54:02.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 -- start start 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fa8104280 0x7f4fa819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 0x7f4fa819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8113d70 con 0x7f4fa8109fd0 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8113bf0 con 0x7f4fa8104280 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fb03ab640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa8113ef0 con 0x7f4fa8109b30 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fae948640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fae948640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59766/0 (socket says 172.21.15.112:59766) 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fae948640 1 -- 172.21.15.112:0/707967386 learned_addr learned my addr 172.21.15.112:0/707967386 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.680+0000 7f4fae147640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fa8104280 0x7f4fa819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fad946640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 0x7f4fa819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fae948640 1 -- 172.21.15.112:0/707967386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 msgr2=0x7f4fa819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fae948640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 0x7f4fa819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fae948640 1 -- 172.21.15.112:0/707967386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fa8104280 msgr2=0x7f4fa819a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fae948640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fa8104280 0x7f4fa819a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fae948640 1 -- 172.21.15.112:0/707967386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa819f570 con 0x7f4fa8109fd0 2024-04-25T16:54:02.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fad946640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 0x7f4fa819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:54:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fae948640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa819efb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400cb00 tx=0x7f4fa400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400adc0 con 0x7f4fa8109fd0 2024-04-25T16:54:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fa4005020 con 0x7f4fa8109fd0 2024-04-25T16:54:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa819f860 con 0x7f4fa8109fd0 2024-04-25T16:54:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400f6d0 con 0x7f4fa8109fd0 2024-04-25T16:54:02.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.681+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa819fdb0 con 0x7f4fa8109fd0 2024-04-25T16:54:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.682+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa806a190 con 0x7f4fa8109fd0 2024-04-25T16:54:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.683+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f4fa40054d0 con 0x7f4fa8109fd0 2024-04-25T16:54:02.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.684+0000 7f4f9b7fe640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4f940afce0 0x7f4f940b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.684+0000 7f4fae147640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4f940afce0 0x7f4f940b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.684+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f4fa40df490 con 0x7f4fa8109fd0 2024-04-25T16:54:02.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.684+0000 7f4fae147640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4f940afce0 0x7f4f940b21f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8104810 tx=0x7f4fa000a2d0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:02.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.688+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fa4014030 con 0x7f4fa8109fd0 2024-04-25T16:54:02.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.866+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fa81a73e0 con 0x7f4fa8109fd0 2024-04-25T16:54:02.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.867+0000 7f4f9b7fe640 1 -- 172.21.15.112:0/707967386 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+23347 (secure 0 0 0) 0x7f4fa40a7d60 con 0x7f4fa8109fd0 2024-04-25T16:54:02.865 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:54:02.866 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":107,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:53:17.159906+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/3947590362":"2024-04-26T16:38:27.501574+0000","172.21.15.112:6800/3947590362":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/386718339":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/4216056022":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/795264311":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/2445069646":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:54:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.869+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4f940afce0 msgr2=0x7f4f940b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.869+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4f940afce0 0x7f4f940b21f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8104810 tx=0x7f4fa000a2d0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.867 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.869+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 msgr2=0x7f4fa819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.869+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa819efb0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400cb00 tx=0x7f4fa400cf00 comp rx=0 tx=0).stop 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 shutdown_connections 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4f940afce0 0x7f4f940b21f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fa8109b30 0x7f4fa819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fa8109fd0 0x7f4fa819efb0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 --2- 172.21.15.112:0/707967386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4fa8104280 0x7f4fa819a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 >> 172.21.15.112:0/707967386 conn(0x7f4fa80ff210 msgr2=0x7f4fa810f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:02.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 shutdown_connections 2024-04-25T16:54:02.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.870+0000 7f4fb03ab640 1 -- 172.21.15.112:0/707967386 wait complete. 2024-04-25T16:54:02.879 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:36:24.204313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:36:25.525471+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:54:02.880 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:54:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 -- 172.21.15.112:0/4091470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 msgr2=0x7fda9c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 --2- 172.21.15.112:0/4091470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c1025b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fda90009900 tx=0x7fda9002a410 comp rx=0 tx=0).stop 2024-04-25T16:54:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 -- 172.21.15.112:0/4091470209 shutdown_connections 2024-04-25T16:54:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 --2- 172.21.15.112:0/4091470209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c102af0 0x7fda9c102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 --2- 172.21.15.112:0/4091470209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 0x7fda9c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 --2- 172.21.15.112:0/4091470209 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c1025b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 -- 172.21.15.112:0/4091470209 >> 172.21.15.112:0/4091470209 conn(0x7fda9c078100 msgr2=0x7fda9c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 -- 172.21.15.112:0/4091470209 shutdown_connections 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.993+0000 7fdaa1292640 1 -- 172.21.15.112:0/4091470209 wait complete. 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 Processor -- start 2024-04-25T16:54:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 -- start start 2024-04-25T16:54:02.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c102af0 0x7fda9c19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c19baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 0x7fda9c19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c112be0 con 0x7fda9c108a20 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c112a60 con 0x7fda9c10d380 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fdaa1292640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c112d60 con 0x7fda9c102af0 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c19baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c19baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59772/0 (socket says 172.21.15.112:59772) 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9a7fc640 1 -- 172.21.15.112:0/501625731 learned_addr learned my addr 172.21.15.112:0/501625731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9b7fe640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 0x7fda9c19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9affd640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c102af0 0x7fda9c19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9a7fc640 1 -- 172.21.15.112:0/501625731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c102af0 msgr2=0x7fda9c19b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.994+0000 7fda9a7fc640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c102af0 0x7fda9c19b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda9a7fc640 1 -- 172.21.15.112:0/501625731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 msgr2=0x7fda9c19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda9a7fc640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 0x7fda9c19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda9a7fc640 1 -- 172.21.15.112:0/501625731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c1a67e0 con 0x7fda9c108a20 2024-04-25T16:54:02.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda9b7fe640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 0x7fda9c19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:54:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda9a7fc640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c19baa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005d00 tx=0x7fda8c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c017070 con 0x7fda9c108a20 2024-04-25T16:54:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda8c009c30 con 0x7fda9c108a20 2024-04-25T16:54:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1a6ad0 con 0x7fda9c108a20 2024-04-25T16:54:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00f620 con 0x7fda9c108a20 2024-04-25T16:54:02.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.995+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a6fd0 con 0x7fda9c108a20 2024-04-25T16:54:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.996+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c06afa0 con 0x7fda9c108a20 2024-04-25T16:54:02.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.997+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fda8c01a070 con 0x7fda9c108a20 2024-04-25T16:54:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.998+0000 7fda83fff640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda840afc60 0x7fda840b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.998+0000 7fda9affd640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda840afc60 0x7fda840b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.998+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7fda8c0deb50 con 0x7fda9c108a20 2024-04-25T16:54:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:02.998+0000 7fda9affd640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda840afc60 0x7fda840b2170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda90031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:02.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.001+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda8c009de0 con 0x7fda9c108a20 2024-04-25T16:54:03.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.184+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fda9c106ee0 con 0x7fda9c108a20 2024-04-25T16:54:03.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.184+0000 7fda83fff640 1 -- 172.21.15.112:0/501625731 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7fda8c0a73a0 con 0x7fda9c108a20 2024-04-25T16:54:03.182 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:54:03.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda840afc60 msgr2=0x7fda840b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda840afc60 0x7fda840b2170 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda90000c00 tx=0x7fda90031400 comp rx=0 tx=0).stop 2024-04-25T16:54:03.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 msgr2=0x7fda9c19baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c19baa0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005d00 tx=0x7fda8c00dde0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 shutdown_connections 2024-04-25T16:54:03.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda840afc60 0x7fda840b2170 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda9c10d380 0x7fda9c19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda9c108a20 0x7fda9c19baa0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 --2- 172.21.15.112:0/501625731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda9c102af0 0x7fda9c19b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 >> 172.21.15.112:0/501625731 conn(0x7fda9c078100 msgr2=0x7fda9c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:03.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 shutdown_connections 2024-04-25T16:54:03.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.186+0000 7fdaa1292640 1 -- 172.21.15.112:0/501625731 wait complete. 2024-04-25T16:54:03.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:54:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 -- 172.21.15.112:0/3174466279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0570108a20 msgr2=0x7f0570102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 --2- 172.21.15.112:0/3174466279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0570108a20 0x7f0570102620 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f056c009900 tx=0x7f056c02a410 comp rx=0 tx=0).stop 2024-04-25T16:54:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 -- 172.21.15.112:0/3174466279 shutdown_connections 2024-04-25T16:54:03.311 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 --2- 172.21.15.112:0/3174466279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570108ec0 0x7f0570102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 --2- 172.21.15.112:0/3174466279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05701031d0 0x7f057010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 --2- 172.21.15.112:0/3174466279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0570108a20 0x7f0570102620 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 -- 172.21.15.112:0/3174466279 >> 172.21.15.112:0/3174466279 conn(0x7f0570078180 msgr2=0x7f05700785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 -- 172.21.15.112:0/3174466279 shutdown_connections 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.310+0000 7f0577fce640 1 -- 172.21.15.112:0/3174466279 wait complete. 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 Processor -- start 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 -- start start 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 0x7f057019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570108a20 0x7f057019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0570108ec0 0x7f057019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0570113d40 con 0x7f05701031d0 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0570113bc0 con 0x7f0570108ec0 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0577fce640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0570113ec0 con 0x7f0570108a20 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0575d6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 0x7f057019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0575d6a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 0x7f057019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59784/0 (socket says 172.21.15.112:59784) 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0575d6a640 1 -- 172.21.15.112:0/969327068 learned_addr learned my addr 172.21.15.112:0/969327068 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.311+0000 7f0575569640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570108a20 0x7f057019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f057656b640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0570108ec0 0x7f057019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0575d6a640 1 -- 172.21.15.112:0/969327068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570108a20 msgr2=0x7f057019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0575d6a640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570108a20 0x7f057019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0575d6a640 1 -- 172.21.15.112:0/969327068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0570108ec0 msgr2=0x7f057019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0575d6a640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0570108ec0 0x7f057019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0575d6a640 1 -- 172.21.15.112:0/969327068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057019f5e0 con 0x7f05701031d0 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f057656b640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0570108ec0 0x7f057019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0575d6a640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 0x7f057019a780 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f056c02a930 tx=0x7f056c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:03.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c03f030 con 0x7f05701031d0 2024-04-25T16:54:03.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f056c003aa0 con 0x7f05701031d0 2024-04-25T16:54:03.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057019f870 con 0x7f05701031d0 2024-04-25T16:54:03.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c035070 con 0x7f05701031d0 2024-04-25T16:54:03.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.312+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f057019fd60 con 0x7f05701031d0 2024-04-25T16:54:03.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.313+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057006a160 con 0x7f05701031d0 2024-04-25T16:54:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.314+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f056c042030 con 0x7f05701031d0 2024-04-25T16:54:03.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.315+0000 7f0566ffd640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f055c0afc60 0x7f055c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.315+0000 7f0575569640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f055c0afc60 0x7f055c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.315+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f056c0ffbb0 con 0x7f05701031d0 2024-04-25T16:54:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.316+0000 7f0575569640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f055c0afc60 0x7f055c0b2170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f057010e4a0 tx=0x7f056000b390 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:03.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.319+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f056c0c8400 con 0x7f05701031d0 2024-04-25T16:54:03.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.496+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0570106f30 con 0x7f05701031d0 2024-04-25T16:54:03.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.497+0000 7f0566ffd640 1 -- 172.21.15.112:0/969327068 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f056c0c7d00 con 0x7f05701031d0 2024-04-25T16:54:03.495 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T16:54:03.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f055c0afc60 msgr2=0x7f055c0b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f055c0afc60 0x7f055c0b2170 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f057010e4a0 tx=0x7f056000b390 comp rx=0 tx=0).stop 2024-04-25T16:54:03.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 msgr2=0x7f057019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 0x7f057019a780 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f056c02a930 tx=0x7f056c004270 comp rx=0 tx=0).stop 2024-04-25T16:54:03.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 shutdown_connections 2024-04-25T16:54:03.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f055c0afc60 0x7f055c0b2170 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0570108a20 0x7f057019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0570108ec0 0x7f057019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 --2- 172.21.15.112:0/969327068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05701031d0 0x7f057019a780 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 >> 172.21.15.112:0/969327068 conn(0x7f0570078180 msgr2=0x7f0570104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 shutdown_connections 2024-04-25T16:54:03.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.499+0000 7f0577fce640 1 -- 172.21.15.112:0/969327068 wait complete. 2024-04-25T16:54:03.506 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:03 smithi112 ceph-mon[29122]: pgmap v497: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-04-25T16:54:03.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:03 smithi112 ceph-mon[29122]: from='client.16677 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:54:03.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:03 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/707967386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:03.507 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:03 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/501625731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:54:03.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:54:03.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 -- 172.21.15.112:0/3781761043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109b10 msgr2=0x7f3e081036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/3781761043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109b10 0x7f3e081036b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009900 tx=0x7f3e0402a410 comp rx=0 tx=0).stop 2024-04-25T16:54:03.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 -- 172.21.15.112:0/3781761043 shutdown_connections 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/3781761043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e08104260 0x7f3e08110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/3781761043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e08109fb0 0x7f3e08103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/3781761043 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109b10 0x7f3e081036b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 -- 172.21.15.112:0/3781761043 >> 172.21.15.112:0/3781761043 conn(0x7f3e080ff210 msgr2=0x7f3e08101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 -- 172.21.15.112:0/3781761043 shutdown_connections 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.621+0000 7f3e0f13f640 1 -- 172.21.15.112:0/3781761043 wait complete. 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.622+0000 7f3e0f13f640 1 Processor -- start 2024-04-25T16:54:03.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.622+0000 7f3e0f13f640 1 -- start start 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.622+0000 7f3e0f13f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e08104260 0x7f3e0819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.622+0000 7f3e0f13f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e08109b10 0x7f3e0819ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0f13f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 0x7f3e0819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0f13f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08113d00 con 0x7f3e08109fb0 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0f13f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08113b80 con 0x7f3e08109b10 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0f13f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3e08113e80 con 0x7f3e08104260 2024-04-25T16:54:03.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 0x7f3e0819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 0x7f3e0819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59798/0 (socket says 172.21.15.112:59798) 2024-04-25T16:54:03.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 -- 172.21.15.112:0/201485379 learned_addr learned my addr 172.21.15.112:0/201485379 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:03.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0cedb640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e08104260 0x7f3e0819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3dfffff640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e08109b10 0x7f3e0819ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 -- 172.21.15.112:0/201485379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e08104260 msgr2=0x7f3e0819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e08104260 0x7f3e0819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 -- 172.21.15.112:0/201485379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e08109b10 msgr2=0x7f3e0819ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e08109b10 0x7f3e0819ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 -- 172.21.15.112:0/201485379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0819f5e0 con 0x7f3e08109fb0 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0d6dc640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 0x7f3e0819efa0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000eb10 tx=0x7f3e0000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00004b20 con 0x7f3e08109fb0 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e00004cc0 con 0x7f3e08109fb0 2024-04-25T16:54:03.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0819f8d0 con 0x7f3e08109fb0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.623+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e00010670 con 0x7f3e08109fb0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.624+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0819fdd0 con 0x7f3e08109fb0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.625+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0806a160 con 0x7f3e08109fb0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.626+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f3e00005000 con 0x7f3e08109fb0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.627+0000 7f3dfdffb640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3df00afc60 0x7f3df00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.627+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7f3e000df820 con 0x7f3e08109fb0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.627+0000 7f3e0cedb640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3df00afc60 0x7f3df00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.627+0000 7f3e0cedb640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3df00afc60 0x7f3df00b2170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e04000c00 tx=0x7f3e04031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:03.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.630+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e00053ff0 con 0x7f3e08109fb0 2024-04-25T16:54:03.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:03 smithi133 ceph-mon[32691]: pgmap v497: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-04-25T16:54:03.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:03 smithi133 ceph-mon[32691]: from='client.16677 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:54:03.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:03 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/707967386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:03.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:03 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/501625731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:54:03.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:03 smithi116 ceph-mon[32712]: pgmap v497: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 465 op/s 2024-04-25T16:54:03.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:03 smithi116 ceph-mon[32712]: from='client.16677 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:54:03.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:03 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/707967386' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:03.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:03 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/501625731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:54:03.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.808+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3e08076060 con 0x7f3e08109fb0 2024-04-25T16:54:03.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.808+0000 7f3dfdffb640 1 -- 172.21.15.112:0/201485379 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f3e0819fdd0 con 0x7f3e08109fb0 2024-04-25T16:54:03.806 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T16:54:03.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.810+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3df00afc60 msgr2=0x7f3df00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.810+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3df00afc60 0x7f3df00b2170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3e04000c00 tx=0x7f3e04031400 comp rx=0 tx=0).stop 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.810+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 msgr2=0x7f3e0819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.810+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 0x7f3e0819efa0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000eb10 tx=0x7f3e0000c2d0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 shutdown_connections 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f3df00afc60 0x7f3df00b2170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3e08109fb0 0x7f3e0819efa0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e08109b10 0x7f3e0819ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 --2- 172.21.15.112:0/201485379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e08104260 0x7f3e0819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 >> 172.21.15.112:0/201485379 conn(0x7f3e080ff210 msgr2=0x7f3e080ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 shutdown_connections 2024-04-25T16:54:03.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.811+0000 7f3e0f13f640 1 -- 172.21.15.112:0/201485379 wait complete. 2024-04-25T16:54:03.819 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T16:54:03.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 -- 172.21.15.112:0/112706046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320104260 msgr2=0x7fa320110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 --2- 172.21.15.112:0/112706046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320104260 0x7fa320110630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009ca0 tx=0x7fa31c02a8c0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 -- 172.21.15.112:0/112706046 shutdown_connections 2024-04-25T16:54:03.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 --2- 172.21.15.112:0/112706046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa320109fb0 0x7fa320103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 --2- 172.21.15.112:0/112706046 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320104260 0x7fa320110630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 --2- 172.21.15.112:0/112706046 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa320109b10 0x7fa3201036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 -- 172.21.15.112:0/112706046 >> 172.21.15.112:0/112706046 conn(0x7fa3200ff1b0 msgr2=0x7fa320101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 -- 172.21.15.112:0/112706046 shutdown_connections 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.929+0000 7fa3274e2640 1 -- 172.21.15.112:0/112706046 wait complete. 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 Processor -- start 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 -- start start 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa320104260 0x7fa32019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa320109b10 0x7fa32019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 0x7fa32019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa320113d00 con 0x7fa320109fb0 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa320113b80 con 0x7fa320104260 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa3274e2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa320113e80 con 0x7fa320109b10 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 0x7fa32019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 0x7fa32019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59822/0 (socket says 172.21.15.112:59822) 2024-04-25T16:54:03.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 -- 172.21.15.112:0/221690490 learned_addr learned my addr 172.21.15.112:0/221690490 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa32527e640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa320104260 0x7fa32019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa324a7d640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa320109b10 0x7fa32019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 -- 172.21.15.112:0/221690490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa320109b10 msgr2=0x7fa32019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa320109b10 0x7fa32019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 -- 172.21.15.112:0/221690490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa320104260 msgr2=0x7fa32019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.930+0000 7fa325a7f640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa320104260 0x7fa32019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:03.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa325a7f640 1 -- 172.21.15.112:0/221690490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32019f4c0 con 0x7fa320109fb0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa325a7f640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 0x7fa32019ef00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006010 tx=0x7fa31c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c003bf0 con 0x7fa320109fb0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa31c003d90 con 0x7fa320109fb0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c02fba0 con 0x7fa320109fb0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32019f750 con 0x7fa320109fb0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.931+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa32019fcf0 con 0x7fa320109fb0 2024-04-25T16:54:03.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.932+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa32006a190 con 0x7fa320109fb0 2024-04-25T16:54:03.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.933+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fa31c02fd40 con 0x7fa320109fb0 2024-04-25T16:54:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.934+0000 7fa3167fc640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa30c0afce0 0x7fa30c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.934+0000 7fa32527e640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa30c0afce0 0x7fa30c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.934+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8600+0+0 (secure 0 0 0) 0x7fa31c100710 con 0x7fa320109fb0 2024-04-25T16:54:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.934+0000 7fa32527e640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa30c0afce0 0x7fa30c0b21f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa318005d00 tx=0x7fa31800b410 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:03.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:03.938+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa31c0c9070 con 0x7fa320109fb0 2024-04-25T16:54:04.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.167+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa32010b690 con 0x7fa320109fb0 2024-04-25T16:54:04.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.176+0000 7fa3167fc640 1 -- 172.21.15.112:0/221690490 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+803666 (secure 0 0 0) 0x7fa31c0c8970 con 0x7fa320109fb0 2024-04-25T16:54:04.184 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:54:04.185 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:54:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "epoch": 25, 2024-04-25T16:54:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "active_gid": 15294, 2024-04-25T16:54:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "active_name": "x", 2024-04-25T16:54:04.185 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addrs": { 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6800", 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 779201591 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:6801", 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 779201591 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "active_addr": "172.21.15.112:6801/779201591", 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "active_change": "2024-04-25T16:38:27.501857+0000", 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T16:54:04.187 INFO:teuthology.orchestra.run.smithi112.stdout: "available": true, 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: "standbys": [ 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 24958, 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "z", 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.193 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.196 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.197 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.197 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.197 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.197 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.197 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.198 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.198 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.198 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.199 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-25T16:54:04.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-25T16:54:04.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-25T16:54:04.203 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.203 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-25T16:54:04.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.204 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-25T16:54:04.204 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.204 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.204 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.205 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-25T16:54:04.205 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-25T16:54:04.205 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.205 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.205 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.205 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-25T16:54:04.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-25T16:54:04.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-25T16:54:04.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.223 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.224 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.225 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T16:54:04.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-25T16:54:04.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.228 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T16:54:04.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.231 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.231 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.232 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.232 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.232 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.232 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.232 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.232 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.233 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.235 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.237 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.238 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.238 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.239 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T16:54:04.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-25T16:54:04.243 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-25T16:54:04.244 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-25T16:54:04.244 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-25T16:54:04.244 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-25T16:54:04.244 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-25T16:54:04.244 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:04.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.253 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-25T16:54:04.254 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T16:54:04.255 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.257 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.258 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-25T16:54:04.259 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T16:54:04.260 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.261 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.262 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.263 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.269 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-25T16:54:04.269 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-25T16:54:04.269 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.271 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-25T16:54:04.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-25T16:54:04.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T16:54:04.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T16:54:04.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.277 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-25T16:54:04.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-25T16:54:04.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T16:54:04.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-25T16:54:04.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.279 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-25T16:54:04.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-25T16:54:04.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:04.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.288 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-25T16:54:04.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-25T16:54:04.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.290 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.291 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-25T16:54:04.293 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.294 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.295 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-25T16:54:04.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.299 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T16:54:04.300 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-25T16:54:04.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.303 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-25T16:54:04.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T16:54:04.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T16:54:04.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-25T16:54:04.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-25T16:54:04.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.314 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.314 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.314 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T16:54:04.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.315 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-25T16:54:04.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-25T16:54:04.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.318 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.319 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.320 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.321 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.322 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.323 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.328 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.330 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.331 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.334 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T16:54:04.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T16:54:04.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-25T16:54:04.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T16:54:04.342 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.343 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-25T16:54:04.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.346 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.346 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T16:54:04.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.350 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.352 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T16:54:04.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.353 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-25T16:54:04.353 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-25T16:54:04.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.354 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.357 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-25T16:54:04.361 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.362 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-25T16:54:04.363 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T16:54:04.364 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.366 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.369 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-25T16:54:04.370 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T16:54:04.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.372 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.373 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.374 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.376 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.377 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.377 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.378 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.381 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.381 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.381 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.382 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-25T16:54:04.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T16:54:04.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.386 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-25T16:54:04.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-25T16:54:04.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-25T16:54:04.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.388 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-25T16:54:04.389 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.389 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.389 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.389 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T16:54:04.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.391 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.392 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-25T16:54:04.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.397 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.398 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T16:54:04.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T16:54:04.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.406 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.407 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-25T16:54:04.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:04.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.421 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.421 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.421 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T16:54:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T16:54:04.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.426 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.426 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.426 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.426 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T16:54:04.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T16:54:04.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.431 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.432 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T16:54:04.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T16:54:04.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.446 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.447 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.450 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T16:54:04.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T16:54:04.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.452 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T16:54:04.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T16:54:04.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-25T16:54:04.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.455 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T16:54:04.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T16:54:04.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.457 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.459 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T16:54:04.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T16:54:04.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.462 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.462 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T16:54:04.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T16:54:04.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.463 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.464 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T16:54:04.465 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.466 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.467 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T16:54:04.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-25T16:54:04.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-25T16:54:04.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.476 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.477 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.478 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.479 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.480 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.483 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.484 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.484 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.484 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.485 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.487 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-25T16:54:04.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-25T16:54:04.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-25T16:54:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T16:54:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.490 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.492 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.493 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.497 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.497 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.498 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.498 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.498 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.498 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-25T16:54:04.498 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-25T16:54:04.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.499 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.499 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.499 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-25T16:54:04.499 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.499 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.499 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.500 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.500 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.500 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.500 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.500 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.500 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.501 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T16:54:04.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-25T16:54:04.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-25T16:54:04.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.508 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.509 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.512 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.513 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-25T16:54:04.516 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.517 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-25T16:54:04.521 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T16:54:04.522 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.523 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:04.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T16:54:04.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-25T16:54:04.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.531 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.532 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.533 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.534 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.534 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.534 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.534 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.535 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.538 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.539 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-25T16:54:04.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:04.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.548 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:04.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.556 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.558 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.559 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.560 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.560 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.560 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.561 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-25T16:54:04.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-25T16:54:04.565 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.565 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.565 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.566 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.567 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.570 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T16:54:04.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T16:54:04.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.570 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.570 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T16:54:04.571 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.572 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-25T16:54:04.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-25T16:54:04.572 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T16:54:04.574 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.574 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-25T16:54:04.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.575 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.576 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.577 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.578 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.579 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.580 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.581 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.582 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.587 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.588 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.589 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.590 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.591 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.592 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.593 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.594 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.595 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.599 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.600 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.602 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.603 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T16:54:04.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.605 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.606 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.608 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.609 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.610 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.611 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.612 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.612 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.612 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.612 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.614 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.614 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.614 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.615 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-25T16:54:04.616 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-25T16:54:04.617 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.619 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.623 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T16:54:04.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T16:54:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.631 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.632 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-25T16:54:04.635 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.636 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T16:54:04.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T16:54:04.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.637 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.637 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.639 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.639 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.639 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.639 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.640 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.642 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.642 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.642 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.642 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.642 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.643 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.644 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.646 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.646 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.646 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.646 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.646 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.647 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.647 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.647 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.647 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.648 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.649 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.650 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.652 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.653 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.654 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.655 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.656 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.659 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.660 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.662 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.663 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.664 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.666 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.667 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.668 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.669 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.669 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.669 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.669 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.669 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.669 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.671 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.671 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.671 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.671 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.671 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.671 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.672 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.673 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.674 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.675 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.676 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.677 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.678 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.678 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.678 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.678 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.679 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.679 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.679 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.681 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.683 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.686 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.687 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.688 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.689 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.690 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.691 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.693 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.697 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.698 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.699 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.702 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.703 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.704 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.705 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.708 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.710 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.711 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.714 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.715 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.716 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.717 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.718 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.719 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.720 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.720 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.720 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.721 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.721 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:04.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.727 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-25T16:54:04.729 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.733 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.734 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.736 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.737 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.738 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-25T16:54:04.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-25T16:54:04.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:04.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-25T16:54:04.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.747 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-25T16:54:04.748 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-25T16:54:04.750 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.751 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.751 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.751 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.752 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.753 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.754 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.754 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.754 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.754 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.755 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.755 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.755 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.755 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.755 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.756 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.757 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.759 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T16:54:04.763 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.764 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.765 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.766 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T16:54:04.767 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:04.768 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T16:54:04.769 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.770 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.771 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T16:54:04.773 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-25T16:54:04.774 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.776 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.777 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.777 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.777 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T16:54:04.777 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.779 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.780 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.781 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.782 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.783 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.784 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.785 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.786 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.788 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-25T16:54:04.788 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-25T16:54:04.788 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.790 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.791 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.793 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.794 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.795 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.796 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.797 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.798 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.799 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.800 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.801 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.802 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.803 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.804 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.805 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.806 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.807 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:04.808 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.809 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.810 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.811 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.812 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.814 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.815 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.816 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.818 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.819 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.820 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.820 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.820 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.821 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.822 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.822 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.822 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.823 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.824 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.824 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.825 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-25T16:54:04.825 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.825 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.825 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.825 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.826 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.827 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.829 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.830 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.831 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.835 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-25T16:54:04.836 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.837 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.839 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.843 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.844 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.844 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.844 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.844 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.844 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.844 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.845 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-25T16:54:04.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-25T16:54:04.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.849 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-25T16:54:04.849 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.849 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-25T16:54:04.850 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-25T16:54:04.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-25T16:54:04.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-25T16:54:04.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.855 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-25T16:54:04.856 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-25T16:54:04.857 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.858 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-25T16:54:04.859 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-25T16:54:04.859 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.859 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.859 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.859 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.859 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.860 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.860 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.861 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.862 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:04.863 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.864 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.865 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.866 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.867 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.868 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-25T16:54:04.869 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.870 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.871 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-25T16:54:04.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.876 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.880 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.881 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.882 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.883 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.884 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.886 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.887 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.887 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.887 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.887 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.889 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.889 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.889 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.889 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.889 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.894 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.895 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.896 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.898 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.898 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.899 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.899 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.899 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.899 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.900 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.902 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.903 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.904 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.905 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.906 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.909 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-25T16:54:04.911 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.912 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.913 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.914 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.915 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.916 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.919 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.920 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.924 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.926 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.929 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-25T16:54:04.929 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-25T16:54:04.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.932 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.932 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T16:54:04.932 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.932 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.933 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-25T16:54:04.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-25T16:54:04.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.933 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.937 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.938 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-25T16:54:04.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.943 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-25T16:54:04.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.951 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.952 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.953 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.954 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.955 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.956 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.957 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.957 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.957 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.959 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.960 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-25T16:54:04.961 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-25T16:54:04.961 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.961 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.961 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.961 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.962 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-25T16:54:04.964 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.965 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.966 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.968 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.969 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.969 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.969 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.969 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.969 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.969 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.970 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.971 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.972 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.973 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.974 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.975 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.976 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.978 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.979 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.980 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.981 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.981 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.981 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.981 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:04.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:04.984 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.988 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T16:54:04.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.993 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:04.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:04.996 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:04.997 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-25T16:54:04.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-25T16:54:04.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:04.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.997 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:04.998 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:04.999 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-25T16:54:04.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-25T16:54:04.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:04.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:04.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:04.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.001 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.001 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.001 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.001 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.002 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.002 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.002 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.002 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.002 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.003 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.003 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.003 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.004 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.006 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.007 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.008 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.009 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-25T16:54:05.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.019 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "gid": 25013, 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "y", 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr_features": 4540701547738038271, 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.020 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.023 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.024 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.025 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.028 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.029 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.030 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-25T16:54:05.031 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-25T16:54:05.032 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-25T16:54:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-25T16:54:05.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-25T16:54:05.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-25T16:54:05.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-25T16:54:05.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-25T16:54:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.042 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-25T16:54:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-25T16:54:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.043 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.044 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.048 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.050 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.051 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-25T16:54:05.052 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T16:54:05.053 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.054 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.055 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T16:54:05.056 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-25T16:54:05.057 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.058 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.059 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.060 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T16:54:05.061 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.063 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.064 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.065 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.066 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.068 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-25T16:54:05.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-25T16:54:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T16:54:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.073 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.074 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-25T16:54:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-25T16:54:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-25T16:54:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-25T16:54:05.075 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-25T16:54:05.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.079 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.080 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.081 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-25T16:54:05.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.089 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.090 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.091 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.093 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-25T16:54:05.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T16:54:05.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.099 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.100 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T16:54:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.101 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.102 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.103 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.109 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-25T16:54:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-25T16:54:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T16:54:05.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-25T16:54:05.113 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T16:54:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-25T16:54:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T16:54:05.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.120 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-25T16:54:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-25T16:54:05.122 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.123 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.124 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.127 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-25T16:54:05.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-25T16:54:05.130 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.131 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.132 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.133 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T16:54:05.134 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.135 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.137 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.138 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.139 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-25T16:54:05.140 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.143 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.144 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-25T16:54:05.145 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.146 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.147 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.148 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.149 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.150 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.151 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.152 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.154 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T16:54:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-25T16:54:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-25T16:54:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.170 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T16:54:05.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-25T16:54:05.172 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-25T16:54:05.173 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T16:54:05.174 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T16:54:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T16:54:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.178 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.180 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T16:54:05.181 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.182 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.183 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.184 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-25T16:54:05.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-25T16:54:05.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-25T16:54:05.192 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.195 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T16:54:05.196 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.197 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.198 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.200 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.201 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.202 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.203 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.204 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.209 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.210 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.211 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.212 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T16:54:05.213 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.214 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.215 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T16:54:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T16:54:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.218 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.219 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-25T16:54:05.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.221 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.222 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.223 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.224 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T16:54:05.225 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.226 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T16:54:05.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.228 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.229 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T16:54:05.232 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.234 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.235 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.239 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.240 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.241 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.243 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.247 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T16:54:05.248 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T16:54:05.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.255 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.256 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T16:54:05.261 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.262 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.263 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.264 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.265 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T16:54:05.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.268 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.269 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.270 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.271 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.274 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.276 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.277 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.280 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T16:54:05.282 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T16:54:05.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-25T16:54:05.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.295 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.296 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.298 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.300 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.301 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.302 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.303 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.304 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.307 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.308 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.310 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.311 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:05.312 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.313 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.314 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.315 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-25T16:54:05.317 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-25T16:54:05.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T16:54:05.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-25T16:54:05.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.324 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.325 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.326 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.327 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.330 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.331 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.332 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.334 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.336 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.337 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-25T16:54:05.338 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.339 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.342 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.343 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.344 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.345 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.346 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.347 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.349 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.350 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.351 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.353 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.354 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.355 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.356 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-25T16:54:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-25T16:54:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T16:54:05.357 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-25T16:54:05.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.361 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.362 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.363 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-25T16:54:05.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-25T16:54:05.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.369 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.370 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.371 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.372 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.373 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.374 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.375 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.376 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.377 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.378 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.379 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.380 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.385 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:05.386 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-25T16:54:05.387 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-25T16:54:05.388 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.389 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.390 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.392 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.393 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.394 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.395 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.396 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.397 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.398 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.399 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.401 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.403 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.404 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.405 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.406 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.407 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.408 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.412 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.413 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.414 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.417 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.419 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.421 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.422 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.423 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.424 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.425 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-25T16:54:05.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T16:54:05.427 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.428 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.429 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.431 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.432 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.433 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.436 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.437 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.438 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.440 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.441 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.442 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-25T16:54:05.445 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.445 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.445 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.445 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T16:54:05.445 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T16:54:05.445 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.446 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.447 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.448 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.449 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.450 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.451 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.452 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.453 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.454 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.455 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.456 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.457 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.458 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.467 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.469 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.470 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.471 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.472 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.473 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.474 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.475 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.476 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.477 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.478 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.482 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.483 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.484 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.485 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.490 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.491 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.492 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.494 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.498 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.499 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.501 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.502 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.503 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.504 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.505 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.506 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.507 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.508 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.510 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.511 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.514 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.515 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.516 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.518 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.520 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.521 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.522 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.523 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.524 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:05.525 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.526 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.527 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.530 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.535 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.536 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.537 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.538 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.539 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.540 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.541 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-25T16:54:05.542 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-25T16:54:05.543 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.544 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.545 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.548 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.549 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.550 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.552 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.553 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.554 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.555 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.556 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.557 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.559 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.560 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-25T16:54:05.561 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-25T16:54:05.562 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.563 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.564 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.565 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.566 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.567 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.568 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-25T16:54:05.569 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T16:54:05.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.571 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T16:54:05.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.573 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.574 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.575 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.576 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.577 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.578 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.579 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.580 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.581 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.582 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.583 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.584 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.585 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.586 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.587 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.588 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.589 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.590 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.591 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.592 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.593 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.594 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.595 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.596 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.597 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.598 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.599 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.600 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:05.601 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.602 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.603 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.604 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.605 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.606 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.607 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.608 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.609 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.610 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.611 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.612 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.613 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.614 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.615 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.616 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.617 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.618 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.619 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.621 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.622 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.623 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.624 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.626 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.627 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.628 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.629 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.630 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.631 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.632 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.633 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.634 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.635 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.636 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-25T16:54:05.637 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.638 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.639 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.640 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.641 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-25T16:54:05.642 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.643 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.644 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-25T16:54:05.645 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.646 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.648 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.649 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-25T16:54:05.650 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.651 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.652 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.653 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.654 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-25T16:54:05.655 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.656 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.657 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.658 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-25T16:54:05.659 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.660 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.661 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.662 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.663 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.664 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.665 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.666 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.667 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.668 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.669 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.670 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.671 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.672 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.673 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.674 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.675 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.676 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.677 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.678 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.679 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.680 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.681 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.682 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.683 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.684 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.685 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.687 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.688 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.689 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.690 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.691 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.692 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.693 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.694 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.695 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.696 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.697 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.698 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.700 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.701 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.702 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.703 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.705 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.706 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.707 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.708 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.709 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-25T16:54:05.710 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T16:54:05.711 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.712 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.713 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.714 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-25T16:54:05.715 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T16:54:05.716 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.717 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.718 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.719 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-25T16:54:05.720 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.721 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.722 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.723 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.724 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.725 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.726 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.727 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-25T16:54:05.728 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.729 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.730 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.731 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.732 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.733 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.734 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.735 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.736 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.737 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.738 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.739 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.740 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-25T16:54:05.741 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.742 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.743 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.744 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.745 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.746 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.747 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.748 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.749 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.750 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.751 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.752 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.753 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.754 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.755 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.756 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.757 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.758 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.759 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.761 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.762 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.763 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.764 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.765 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.766 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.767 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.768 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.769 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T16:54:05.770 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.771 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.772 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.773 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-25T16:54:05.774 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.775 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.776 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.777 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.778 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.779 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.780 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.781 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.782 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.783 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.784 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.785 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.786 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.787 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.788 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.789 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.790 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.791 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.792 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.793 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: "modules": [ 2024-04-25T16:54:05.794 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard", 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "iostat", 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "nfs", 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "restful", 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "snap_schedule" 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "available_modules": [ 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.795 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "alerts", 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.796 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.797 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.798 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.799 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.800 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.801 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.803 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.804 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.805 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_destination": { 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_destination", 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.806 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email address to send alerts to", 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_from_name": { 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_from_name", 2024-04-25T16:54:05.807 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "Ceph", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Email From: name", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.808 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_host": { 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_host", 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.809 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP server", 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_password": { 2024-04-25T16:54:05.810 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_password", 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.811 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Password to authenticate with", 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_port": { 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_port", 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.812 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "465", 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP port", 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.813 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_sender": { 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_sender", 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.814 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SMTP envelope sender", 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_ssl": { 2024-04-25T16:54:05.815 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_ssl", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T16:54:05.816 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "smtp_user": { 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "smtp_user", 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.817 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "User to authenticate as", 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.818 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.819 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.820 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "balancer", 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "active": { 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "active", 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.821 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.822 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_time": { 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_time", 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0000", 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.823 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "begin_weekday": { 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "begin_weekday", 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.824 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.825 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_max_iterations": { 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_max_iterations", 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "25", 2024-04-25T16:54:05.826 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "250", 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_metrics": { 2024-04-25T16:54:05.827 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_metrics", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T16:54:05.828 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "crush_compat_step": { 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crush_compat_step", 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.829 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.5", 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0.001", 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "0.999", 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "aggressiveness of optimization", 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.830 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "end_time": { 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_time", 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2359", 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.831 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "end_weekday": { 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "end_weekday", 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.832 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "0", 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "6", 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.833 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.834 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.835 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.836 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.837 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.838 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.839 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.840 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.841 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "min_score": { 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_score", 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.842 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.843 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "upmap", 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.844 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "crush-compat", 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "none", 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "read", 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap", 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap-read" 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Balancer mode", 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.845 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_ids": { 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_ids", 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.846 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.847 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.848 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.849 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.850 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_deviation": { 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_deviation", 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:05.851 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "upmap_max_optimizations": { 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "upmap_max_optimizations", 2024-04-25T16:54:05.852 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.853 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm", 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.854 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_down_multiplier": { 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_down_multiplier", 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.855 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_refresh_rate": { 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_refresh_rate", 2024-04-25T16:54:05.856 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "20", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.857 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "agent_starting_port": { 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "agent_starting_port", 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.858 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4721", 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.859 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_ptrace": { 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_ptrace", 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.860 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_interval": { 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_interval", 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.861 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.862 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "autotune_memory_target_ratio": { 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0.7", 2024-04-25T16:54:05.863 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm_log_destination": { 2024-04-25T16:54:05.864 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cephadm_log_destination", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "file", 2024-04-25T16:54:05.865 INFO:teuthology.orchestra.run.smithi112.stdout: "file,syslog", 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "syslog" 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "cgroups_split": { 2024-04-25T16:54:05.866 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cgroups_split", 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.867 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "config_checks_enabled": { 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_checks_enabled", 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.868 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.869 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "config_dashboard": { 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_dashboard", 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.870 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_alertmanager": { 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_alertmanager", 2024-04-25T16:54:05.871 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.872 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_base": { 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_base", 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T16:54:05.873 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Container image name, without the tag", 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.874 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_elasticsearch": { 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_elasticsearch", 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.875 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "elasticsearch container image", 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_grafana": { 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_grafana", 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.876 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.877 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_haproxy": { 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_haproxy", 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.878 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "HAproxy container image", 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_agent": { 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_agent", 2024-04-25T16:54:05.879 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger agent container image", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.880 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_collector": { 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_collector", 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T16:54:05.881 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger collector container image", 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.882 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_jaeger_query": { 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_jaeger_query", 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.883 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Jaeger query container image", 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_keepalived": { 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_keepalived", 2024-04-25T16:54:05.884 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Keepalived container image", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.885 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_loki": { 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_loki", 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.886 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Loki container image", 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.887 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_node_exporter": { 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_node_exporter", 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.888 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_nvmeof": { 2024-04-25T16:54:05.889 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_nvmeof", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Nvme-of container image", 2024-04-25T16:54:05.890 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_prometheus": { 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_prometheus", 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.891 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Prometheus container image", 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.892 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_promtail": { 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_promtail", 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.893 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Promtail container image", 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_samba": { 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_samba", 2024-04-25T16:54:05.894 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Samba/SMB container image", 2024-04-25T16:54:05.895 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "container_image_snmp_gateway": { 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_image_snmp_gateway", 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.896 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "SNMP Gateway container image", 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.897 INFO:teuthology.orchestra.run.smithi112.stdout: "container_init": { 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "container_init", 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.898 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "daemon_cache_timeout": { 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "daemon_cache_timeout", 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.899 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.900 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_cephadm_command_timeout": { 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "900", 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.901 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "default_registry": { 2024-04-25T16:54:05.902 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "default_registry", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "docker.io", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T16:54:05.903 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "device_cache_timeout": { 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_cache_timeout", 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.904 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1800", 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache device inventory", 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.905 INFO:teuthology.orchestra.run.smithi112.stdout: "device_enhanced_scan": { 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_enhanced_scan", 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.906 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "facts_cache_timeout": { 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "facts_cache_timeout", 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.907 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.908 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.908 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:05.908 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "seconds to cache host facts data", 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.910 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "grafana_dashboards_path": { 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "grafana_dashboards_path", 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.911 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "host_check_interval": { 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "host_check_interval", 2024-04-25T16:54:05.912 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to perform a host check", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.913 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "hw_monitoring": { 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hw_monitoring", 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.914 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.915 INFO:teuthology.orchestra.run.smithi112.stdout: "inventory_list_all": { 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "inventory_list_all", 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.916 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.917 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.918 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.919 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "log_refresh_metadata": { 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_refresh_metadata", 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.920 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.921 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.922 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.923 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.924 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.925 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.926 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.927 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T16:54:05.928 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "*", 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.929 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "max_count_per_host": { 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_count_per_host", 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.930 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of daemons per service per host", 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.931 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max_osd_draining_count": { 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_osd_draining_count", 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.932 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "migration_current": { 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "migration_current", 2024-04-25T16:54:05.933 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "internal - do not modify", 2024-04-25T16:54:05.934 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "mode": { 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mode", 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.935 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "root", 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm-package", 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "root" 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.936 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "oob_default_addr": { 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "oob_default_addr", 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.937 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "169.254.1.1", 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.938 INFO:teuthology.orchestra.run.smithi112.stdout: "prometheus_alerts_path": { 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus_alerts_path", 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.939 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_insecure": { 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_insecure", 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.940 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.941 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_password": { 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_password", 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.942 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_url": { 2024-04-25T16:54:05.943 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_url", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T16:54:05.944 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "registry_username": { 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "registry_username", 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.945 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.946 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "secure_monitoring_stack": { 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "secure_monitoring_stack", 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.947 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "service_discovery_port": { 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "service_discovery_port", 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.948 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8765", 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "cephadm service discovery port", 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.949 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.950 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.951 INFO:teuthology.orchestra.run.smithi112.stdout: "ssh_config_file": { 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssh_config_file", 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.952 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "use_agent": { 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_agent", 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.953 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.954 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "use_repo_digest": { 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "use_repo_digest", 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.955 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_failed_host_check": { 2024-04-25T16:54:05.956 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_failed_host_check", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T16:54:05.957 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.958 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.958 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.958 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.958 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_daemons": { 2024-04-25T16:54:05.958 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_daemons", 2024-04-25T16:54:05.958 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.959 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_on_stray_hosts": { 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_on_stray_hosts", 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.960 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.961 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crash", 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:05.962 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.963 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.964 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.965 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.966 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:05.967 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.968 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.969 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.970 INFO:teuthology.orchestra.run.smithi112.stdout: "retain_interval": { 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retain_interval", 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "31536000", 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.971 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:05.972 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.973 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_recent_interval": { 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_recent_interval", 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1209600", 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.974 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:05.975 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dashboard", 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:05.976 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.977 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.978 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T16:54:05.979 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.980 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_ENABLED": { 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.981 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.982 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.983 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.984 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.985 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.986 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.987 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.988 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.989 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.990 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.991 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T16:54:05.992 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.993 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.994 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.995 INFO:teuthology.orchestra.run.smithi112.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.996 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.997 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:05.998 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:05.999 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T16:54:06.000 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.001 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_URL": { 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_URL", 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.002 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.003 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.004 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T16:54:06.005 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.006 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.007 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.008 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.009 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T16:54:06.010 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.011 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.012 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "[]", 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.013 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "{}", 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.014 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T16:54:06.015 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.016 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.017 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.018 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.019 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.020 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.021 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.022 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T16:54:06.023 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.024 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.025 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.026 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.027 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T16:54:06.028 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.029 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.030 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.031 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.032 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T16:54:06.033 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.034 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8", 2024-04-25T16:54:06.035 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.036 INFO:teuthology.orchestra.run.smithi112.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "45", 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.037 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.038 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.039 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "admin", 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T16:54:06.041 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.042 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.043 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.044 INFO:teuthology.orchestra.run.smithi112.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.045 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.046 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.047 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:06.048 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.049 INFO:teuthology.orchestra.run.smithi112.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.050 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "cross_origin_url": { 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cross_origin_url", 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.051 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.052 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "crt_file": { 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "crt_file", 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.053 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "debug": { 2024-04-25T16:54:06.054 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "debug", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Enable/disable debug options", 2024-04-25T16:54:06.055 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "jwt_token_ttl": { 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "jwt_token_ttl", 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.056 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "28800", 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.057 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.058 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.059 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.060 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.061 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.062 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.063 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.064 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.065 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.066 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "motd": { 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "motd", 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.068 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "The message of the day", 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect_resolve_ip_addr": { 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T16:54:06.069 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.070 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.071 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.072 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8080", 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.073 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.074 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.075 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.076 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.077 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl_server_port": { 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl_server_port", 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8443", 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.078 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T16:54:06.079 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "redirect", 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.080 INFO:teuthology.orchestra.run.smithi112.stdout: "redirect" 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T16:54:06.081 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.082 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "url_prefix": { 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url_prefix", 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.083 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.084 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.085 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_monitoring": { 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_monitoring", 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.086 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "monitor device health metrics", 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.087 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.088 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.089 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.090 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.091 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.092 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.093 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.094 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.095 INFO:teuthology.orchestra.run.smithi112.stdout: "mark_out_threshold": { 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mark_out_threshold", 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "2419200", 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.096 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "pool_name": { 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pool_name", 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.097 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "device_health_metrics", 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.098 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "retention_period": { 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "retention_period", 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15552000", 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.099 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long to retain device health metrics", 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_frequency": { 2024-04-25T16:54:06.100 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_frequency", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T16:54:06.101 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "self_heal": { 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "self_heal", 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.102 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.103 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.104 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.105 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "warn_threshold": { 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "warn_threshold", 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7257600", 2024-04-25T16:54:06.107 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.108 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "diskprediction_local", 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.109 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.110 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.111 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.112 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.113 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.114 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.115 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.116 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.117 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "predict_interval": { 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predict_interval", 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.118 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "86400", 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.119 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "predictor_model": { 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "predictor_model", 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "prophetstor", 2024-04-25T16:54:06.120 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:06.121 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "600", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.122 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.123 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.124 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "influx", 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": false, 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "influxdb python module not found", 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.125 INFO:teuthology.orchestra.run.smithi112.stdout: "batch_size": { 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "batch_size", 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5000", 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.126 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "database": { 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "database", 2024-04-25T16:54:06.127 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "ceph", 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T16:54:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": { 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "hostname", 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.129 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server hostname", 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.130 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "30", 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "5", 2024-04-25T16:54:06.131 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.132 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.133 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.134 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.135 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.136 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.137 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.138 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.139 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.140 INFO:teuthology.orchestra.run.smithi112.stdout: "password": { 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "password", 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.141 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "password of InfluxDB server user", 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "port": { 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "port", 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.142 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "8086", 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "InfluxDB server port", 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.143 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.144 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "ssl": { 2024-04-25T16:54:06.145 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ssl", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "false", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T16:54:06.146 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "threads": { 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threads", 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.147 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "32", 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.148 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "username": { 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "username", 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.149 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "username of InfluxDB server user", 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "verify_ssl": { 2024-04-25T16:54:06.150 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "verify_ssl", 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "true", 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.151 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "insights", 2024-04-25T16:54:06.152 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.153 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.154 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.155 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.156 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.157 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.158 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.158 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.158 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.158 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.158 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.158 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.159 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.160 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.161 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.162 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "iostat", 2024-04-25T16:54:06.163 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.164 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.165 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.166 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.167 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.168 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.169 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.170 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.171 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.172 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.173 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "k8sevents", 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.174 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_event_retention_days": { 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "ceph_event_retention_days", 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.175 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "7", 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.176 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "config_check_secs": { 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "config_check_secs", 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "10", 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.177 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.178 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.179 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.180 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.181 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.182 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.183 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.184 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.185 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.186 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.187 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.188 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "localpool", 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "failure_domain": { 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "failure_domain", 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.189 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "host", 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "failure domain for any created local pool", 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.190 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.191 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.192 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.193 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.194 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.195 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.196 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.197 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.198 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min_size": { 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "min_size", 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.199 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default min_size for any created local pool", 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "num_rep": { 2024-04-25T16:54:06.200 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "num_rep", 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3", 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.201 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default replica count for any created local pool", 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_num": { 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_num", 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.202 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "128", 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.203 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "prefix": { 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prefix", 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.204 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "name prefix for any created local pool", 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.205 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.206 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "subtree": { 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "subtree", 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.207 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "rack", 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T16:54:06.208 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mds_autoscaler", 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.209 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.210 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.211 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.212 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.213 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.214 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.215 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.216 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.217 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.218 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.219 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirroring", 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.220 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.221 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.222 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.223 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.224 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.225 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.226 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.227 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.228 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.229 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "nfs", 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.230 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.231 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.232 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.233 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.234 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.235 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.236 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.237 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.238 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.239 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.240 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "fail_fs": { 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "fail_fs", 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.241 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.242 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.243 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.244 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.245 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.246 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.247 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.248 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.249 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.250 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator": { 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "orchestrator", 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.251 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "cephadm", 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "rook", 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "test_orchestrator" 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Orchestrator backend", 2024-04-25T16:54:06.252 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.253 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.254 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_perf_query", 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.255 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.256 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.257 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.258 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.259 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.260 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.261 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.262 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.263 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.264 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "osd_support", 2024-04-25T16:54:06.265 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.266 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.267 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.268 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.269 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.270 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.271 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.272 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.273 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.274 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.275 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "pg_autoscaler", 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.276 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.277 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.278 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.279 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.280 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.281 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.282 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.283 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.284 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.285 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.286 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.287 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "threshold": { 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "threshold", 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.288 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "3.0", 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1.0", 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "scaling threshold", 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.289 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "progress", 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.290 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_pg_recovery_event": { 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_pg_recovery_event", 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.291 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.292 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.293 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.294 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.295 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.296 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.297 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.298 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.299 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.300 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.301 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "max_completed_events": { 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_completed_events", 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.302 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "50", 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "number of past completed events to remember", 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.303 INFO:teuthology.orchestra.run.smithi112.stdout: "sleep_interval": { 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sleep_interval", 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "5", 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.304 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "how long the module is going to sleep", 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.305 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.306 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "prometheus", 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.307 INFO:teuthology.orchestra.run.smithi112.stdout: "cache": { 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "cache", 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.308 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "exclude_perf_counters": { 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "exclude_perf_counters", 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.309 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T16:54:06.310 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.311 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.311 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.311 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.311 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.311 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.312 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.312 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.312 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.312 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.312 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.312 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.313 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.313 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.313 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.313 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.313 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.314 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.314 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.314 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.314 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.314 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.314 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.315 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.316 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.317 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.318 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.319 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools": { 2024-04-25T16:54:06.320 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools", 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.321 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.322 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "300", 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.323 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "scrape_interval": { 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "scrape_interval", 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15.0", 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.324 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:06.325 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "::", 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T16:54:06.326 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.327 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "9283", 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.328 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.329 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "stale_cache_strategy": { 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stale_cache_strategy", 2024-04-25T16:54:06.330 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "log", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.331 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_behaviour": { 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_behaviour", 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.332 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "default", 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "default", 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "error" 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.333 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "standby_error_status_code": { 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "standby_error_status_code", 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.334 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "500", 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "400", 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "599", 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.335 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.336 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.337 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.338 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.339 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.340 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.341 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.342 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.343 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.344 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_snap_create": { 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_snap_create", 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10", 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.345 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "mirror_snapshot_schedule": { 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.346 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.347 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.348 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.349 INFO:teuthology.orchestra.run.smithi112.stdout: "trash_purge_schedule": { 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "trash_purge_schedule", 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.350 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "restful", 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "enable_auth": { 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enable_auth", 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.352 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.353 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "key_file": { 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "key_file", 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.354 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.355 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.356 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.357 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.358 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.359 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.360 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.361 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.362 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.363 INFO:teuthology.orchestra.run.smithi112.stdout: "server_addr": { 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_addr", 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.364 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "server_port": { 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "server_port", 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.365 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.366 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.367 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.368 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rgw", 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.369 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.370 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.371 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.372 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.373 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.374 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.375 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.376 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.377 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rook", 2024-04-25T16:54:06.379 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.380 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.381 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.382 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.383 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.384 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.385 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.386 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.387 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.388 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.389 INFO:teuthology.orchestra.run.smithi112.stdout: "storage_class": { 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "storage_class", 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "local", 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.390 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "selftest", 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.392 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.393 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.394 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.395 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.396 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.397 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.398 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.399 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "roption1": { 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption1", 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.400 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.401 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "roption2": { 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "roption2", 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "xyz", 2024-04-25T16:54:06.402 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption1": { 2024-04-25T16:54:06.403 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption1", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.404 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption2": { 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption2", 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.405 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption3": { 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption3", 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "float", 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.407 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption4": { 2024-04-25T16:54:06.408 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption4", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption5": { 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption5", 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.410 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption6": { 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption6", 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.412 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "rwoption7": { 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rwoption7", 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.413 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "1", 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "42", 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.414 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.415 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.416 INFO:teuthology.orchestra.run.smithi112.stdout: "testkey": { 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testkey", 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.417 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "testlkey": { 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testlkey", 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.418 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.419 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "testnewline": { 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "testnewline", 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.420 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.421 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "allow_m_granularity": { 2024-04-25T16:54:06.422 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "allow_m_granularity", 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.423 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "dump_on_update": { 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "dump_on_update", 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.424 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "dump database to debug log on update", 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.425 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.426 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.427 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.428 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.429 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.430 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.431 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.432 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.433 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.434 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.435 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "stats", 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.436 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.437 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.438 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.439 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.440 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.441 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.442 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.443 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.444 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.445 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.446 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "status", 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.447 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.448 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.449 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.450 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.451 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.452 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.453 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.454 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.455 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.456 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.457 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telegraf", 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "address": { 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "address", 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.458 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.459 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "15", 2024-04-25T16:54:06.460 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.461 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.462 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.463 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.464 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.465 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.466 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.467 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.468 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.469 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.470 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.471 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "telemetry", 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_basic": { 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_basic", 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.472 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.473 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_crash": { 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_crash", 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.474 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_device": { 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_device", 2024-04-25T16:54:06.475 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.476 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_ident": { 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_ident", 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.477 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.478 INFO:teuthology.orchestra.run.smithi112.stdout: "channel_perf": { 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "channel_perf", 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.479 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "contact": { 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "contact", 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.480 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.481 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "description": { 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "description", 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.482 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "device_url": { 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "device_url", 2024-04-25T16:54:06.483 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.484 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "enabled": { 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "enabled", 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.485 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:06.486 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "24", 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "8", 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.487 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "last_opt_revision": { 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "last_opt_revision", 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.488 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "1", 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.489 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard": { 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard", 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.490 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "leaderboard_description": { 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "leaderboard_description", 2024-04-25T16:54:06.491 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.492 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.493 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.494 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.495 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.496 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.497 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.498 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.499 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.500 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "organization": { 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "organization", 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.501 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.502 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "proxy": { 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "proxy", 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.503 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.504 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.505 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "url": { 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "url", 2024-04-25T16:54:06.506 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.507 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "test_orchestrator", 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.508 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.509 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.510 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.511 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.512 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.513 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.514 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.515 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.516 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.517 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.518 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.519 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.519 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.519 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T16:54:06.519 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.522 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.523 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.524 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.525 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.526 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.527 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.528 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.529 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "max_concurrent_clones": { 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "max_concurrent_clones", 2024-04-25T16:54:06.530 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "4", 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T16:54:06.531 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "periodic_async_work": { 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "periodic_async_work", 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Periodically check for async work", 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.533 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_delay": { 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_delay", 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.534 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "snapshot_clone_no_wait": { 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.535 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "True", 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.536 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.537 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.538 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix", 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: "can_run": true, 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: "error_string": "", 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: "module_options": { 2024-04-25T16:54:06.539 INFO:teuthology.orchestra.run.smithi112.stdout: "discovery_interval": { 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "discovery_interval", 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "uint", 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "100", 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.540 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "identifier": { 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "identifier", 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.541 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.542 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "interval": { 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "interval", 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "secs", 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "60", 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.543 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "log_level": { 2024-04-25T16:54:06.544 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_level", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.545 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.546 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster": { 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster", 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.547 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_cluster_level": { 2024-04-25T16:54:06.548 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_cluster_level", 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "info", 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [ 2024-04-25T16:54:06.549 INFO:teuthology.orchestra.run.smithi112.stdout: "", 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "critical", 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "debug", 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "error", 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "info", 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "warning" 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.550 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "log_to_file": { 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "log_to_file", 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "bool", 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.551 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "False", 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.552 INFO:teuthology.orchestra.run.smithi112.stdout: "sqlite3_killpoint": { 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "sqlite3_killpoint", 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "dev", 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 1, 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "0", 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.553 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_host": { 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_host", 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.554 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "", 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.555 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_port": { 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_port", 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "int", 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "10051", 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.556 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "zabbix_sender": { 2024-04-25T16:54:06.557 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "zabbix_sender", 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "str", 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "level": "advanced", 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "flags": 0, 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "min": "", 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "max": "", 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "enum_allowed": [], 2024-04-25T16:54:06.558 INFO:teuthology.orchestra.run.smithi112.stdout: "desc": "", 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: "long_desc": "", 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: "tags": [], 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: "see_also": [] 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.559 INFO:teuthology.orchestra.run.smithi112.stdout: "services": { 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "dashboard": "https://172.21.15.112:8443/" 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "always_on_modules": { 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "octopus": [ 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T16:54:06.560 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "pacific": [ 2024-04-25T16:54:06.561 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T16:54:06.562 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "quincy": [ 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T16:54:06.563 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "reef": [ 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T16:54:06.564 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T16:54:06.565 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: ], 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "squid": [ 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "balancer", 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "crash", 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "devicehealth", 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "orchestrator", 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "pg_autoscaler", 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "progress", 2024-04-25T16:54:06.566 INFO:teuthology.orchestra.run.smithi112.stdout: "rbd_support", 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: "status", 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: "telemetry", 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: "volumes" 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: "last_failure_osd_epoch": 80, 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: "active_clients": [ 2024-04-25T16:54:06.567 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "devicehealth", 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1958422716 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.568 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "libcephsqlite", 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3230768954 2024-04-25T16:54:06.569 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "rbd_support", 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:06.570 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 2314532912 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.571 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1977204904 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "snap_schedule", 2024-04-25T16:54:06.572 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1225918208 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.573 INFO:teuthology.orchestra.run.smithi112.stdout: "name": "volumes", 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: "addrvec": [ 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v2", 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "172.21.15.112:0", 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 1438803038 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:06.574 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T16:54:06.575 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:54:06.576 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:04 smithi116 ceph-mon[32712]: from='client.16683 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:54:06.576 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:04 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/969327068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:54:06.576 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:04 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/201485379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:54:06.576 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:04 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/221690490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T16:54:06.577 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:05 smithi116 ceph-mon[32712]: pgmap v498: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 505 op/s 2024-04-25T16:54:06.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:04 smithi133 ceph-mon[32691]: from='client.16683 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:54:06.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:04 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/969327068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:54:06.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:04 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/201485379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:54:06.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:04 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/221690490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T16:54:06.577 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:05 smithi133 ceph-mon[32691]: pgmap v498: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 505 op/s 2024-04-25T16:54:06.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa30c0afce0 msgr2=0x7fa30c0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:06.578 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa30c0afce0 0x7fa30c0b21f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa318005d00 tx=0x7fa31800b410 comp rx=0 tx=0).stop 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 msgr2=0x7fa32019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 0x7fa32019ef00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006010 tx=0x7fa31c02f380 comp rx=0 tx=0).stop 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 shutdown_connections 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fa30c0afce0 0x7fa30c0b21f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa320109b10 0x7fa32019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa320109fb0 0x7fa32019ef00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 --2- 172.21.15.112:0/221690490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa320104260 0x7fa32019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:06.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 >> 172.21.15.112:0/221690490 conn(0x7fa3200ff1b0 msgr2=0x7fa320106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 shutdown_connections 2024-04-25T16:54:06.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:04.181+0000 7fa3274e2640 1 -- 172.21.15.112:0/221690490 wait complete. 2024-04-25T16:54:06.586 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T16:54:06.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T16:54:06.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:04 smithi112 ceph-mon[29122]: from='client.16683 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:54:06.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:04 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/969327068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:54:06.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:04 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/201485379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:54:06.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:04 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/221690490' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T16:54:06.587 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:05 smithi112 ceph-mon[29122]: pgmap v498: 97 pgs: 97 active+clean; 303 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.1 MiB/s wr, 505 op/s 2024-04-25T16:54:06.999 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:54:07.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:07 smithi112 ceph-mon[29122]: pgmap v499: 97 pgs: 97 active+clean; 305 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-04-25T16:54:07.414 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:07 smithi112 ceph-mon[29122]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T16:54:07.559 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:07 smithi116 ceph-mon[32712]: pgmap v499: 97 pgs: 97 active+clean; 305 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-04-25T16:54:07.560 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:07 smithi116 ceph-mon[32712]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T16:54:07.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:07 smithi133 ceph-mon[32691]: pgmap v499: 97 pgs: 97 active+clean; 305 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.4 MiB/s wr, 429 op/s 2024-04-25T16:54:07.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:07 smithi133 ceph-mon[32691]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T16:54:09.072 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3040, 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T16:54:09.073 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001607120, 2024-04-25T16:54:09.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000033161, 2024-04-25T16:54:09.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000569121, 2024-04-25T16:54:09.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:09.074 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:09.074 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-04-25T16:54:09.075 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000861133, 2024-04-25T16:54:09.075 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000287044 2024-04-25T16:54:09.075 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.075 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:09.075 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:09.076 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:09.076 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:09.076 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.076 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T16:54:09.077 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T16:54:09.077 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.077 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:09.077 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3, 2024-04-25T16:54:09.078 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 220, 2024-04-25T16:54:09.078 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 480, 2024-04-25T16:54:09.079 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 274784, 2024-04-25T16:54:09.079 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-25T16:54:09.079 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T16:54:09.079 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.015664557, 2024-04-25T16:54:09.079 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.010934367, 2024-04-25T16:54:09.079 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002281209, 2024-04-25T16:54:09.080 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:09.080 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:09.080 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 217, 2024-04-25T16:54:09.080 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012001171, 2024-04-25T16:54:09.080 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000055304 2024-04-25T16:54:09.080 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.081 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:09.081 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:09.081 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:09.081 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:09.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.082 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T16:54:09.082 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 274784 2024-04-25T16:54:09.082 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.082 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 295, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 285, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1560800, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 176064, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.048519802, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.014634121, 2024-04-25T16:54:09.083 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.038913423, 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005201725, 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 284, 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.015121219, 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000053243 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:09.084 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1560800, 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 176064 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T16:54:09.085 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:09.086 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:09.087 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:09.088 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T16:54:09.089 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 147970, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 29, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T16:54:09.090 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:09.091 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T16:54:09.092 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 298, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1519139, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:09.093 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 298, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 298, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1519139, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:09.094 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:09.095 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:09.095 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:54:09.344 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:09 smithi112 ceph-mon[29122]: pgmap v501: 97 pgs: 97 active+clean; 305 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2024-04-25T16:54:09.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:09 smithi133 ceph-mon[32691]: pgmap v501: 97 pgs: 97 active+clean; 305 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2024-04-25T16:54:09.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:09 smithi116 ceph-mon[32712]: pgmap v501: 97 pgs: 97 active+clean; 305 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 4.7 MiB/s wr, 270 op/s 2024-04-25T16:54:10.115 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T16:54:10.116 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T16:54:10.116 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T16:54:10.116 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T16:54:10.116 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T16:54:10.116 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T16:54:10.116 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T16:54:10.116 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T16:54:10.117 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T16:54:10.461 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:54:11.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:11 smithi112 ceph-mon[29122]: pgmap v502: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 300 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 168 op/s 2024-04-25T16:54:11.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:11 smithi133 ceph-mon[32691]: pgmap v502: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 300 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 168 op/s 2024-04-25T16:54:11.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:11 smithi116 ceph-mon[32712]: pgmap v502: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 300 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 168 op/s 2024-04-25T16:54:12.694 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:54:12.694 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:12.694 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 0, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 2, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 0, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 0, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001271978, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T16:54:12.695 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000296017, 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:12.696 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 307, 2024-04-25T16:54:12.697 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 295, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1566704, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 180848, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.050579856, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.015750333, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.039750119, 2024-04-25T16:54:12.698 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005222505, 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 295, 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.011792291, 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000039973 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:12.699 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1566704, 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 180848 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:12.700 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 224, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 480, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 275488, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.015285240, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.010517950, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002243085, 2024-04-25T16:54:12.701 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 221, 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.010331111, 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000046747 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:12.702 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 275488 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T16:54:12.703 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:12.704 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:12.705 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:12.706 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T16:54:12.707 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148714, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T16:54:12.708 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:12.709 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:12.710 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 306, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1520880, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 306, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T16:54:12.711 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 306, 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1520880, 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:12.712 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T16:54:13.272 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T16:54:13.273 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T16:54:13.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T16:54:13.514 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:54:13.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:13 smithi112 ceph-mon[29122]: pgmap v503: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 300 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 168 op/s 2024-04-25T16:54:13.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:13 smithi133 ceph-mon[32691]: pgmap v503: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 300 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 168 op/s 2024-04-25T16:54:13.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:13 smithi116 ceph-mon[32712]: pgmap v503: 97 pgs: 14 active+clean+snaptrim, 30 active+clean+snaptrim_wait, 53 active+clean; 300 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 168 op/s 2024-04-25T16:54:15.439 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:54:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-04-25T16:54:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-04-25T16:54:15.439 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3040, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.001532570, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000039323, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000619482, 2024-04-25T16:54:15.440 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-04-25T16:54:15.441 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000873395, 2024-04-25T16:54:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000291131 2024-04-25T16:54:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:15.442 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:15.443 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 300, 2024-04-25T16:54:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 290, 2024-04-25T16:54:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1562848, 2024-04-25T16:54:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 178480, 2024-04-25T16:54:15.444 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.049721671, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.014964906, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.040462732, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005201113, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 289, 2024-04-25T16:54:15.445 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012022934, 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000041601 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.446 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1562848, 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 178480 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3, 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 222, 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 480, 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 275136, 2024-04-25T16:54:15.447 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.015697453, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.010277237, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.003040650, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 219, 2024-04-25T16:54:15.448 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.012779929, 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000058355 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T16:54:15.449 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 275136 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:15.450 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:15.451 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:15.452 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:15.453 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:15.454 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:15.455 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148731, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 31, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:15.456 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:15.457 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:15.458 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 303, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1520490, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 303, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T16:54:15.459 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 303, 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1520490, 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:15.460 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:54:15.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:15 smithi112 ceph-mon[29122]: pgmap v504: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 299 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 140 op/s 2024-04-25T16:54:15.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:15 smithi133 ceph-mon[32691]: pgmap v504: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 299 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 140 op/s 2024-04-25T16:54:15.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:15 smithi116 ceph-mon[32712]: pgmap v504: 97 pgs: 9 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 66 active+clean; 299 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.0 MiB/s wr, 140 op/s 2024-04-25T16:54:16.204 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T16:54:16.204 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T16:54:16.204 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T16:54:16.205 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T16:54:16.205 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T16:54:16.205 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T16:54:16.205 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T16:54:16.205 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T16:54:16.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T16:54:16.468 INFO:teuthology.orchestra.run.smithi112.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T16:54:17.562 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:17 smithi112 ceph-mon[29122]: pgmap v505: 97 pgs: 97 active+clean; 298 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-04-25T16:54:17.562 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:17 smithi112 ceph-mon[29122]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T16:54:17.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:17 smithi133 ceph-mon[32691]: pgmap v505: 97 pgs: 97 active+clean; 298 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-04-25T16:54:17.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:17 smithi133 ceph-mon[32691]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T16:54:17.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:17 smithi116 ceph-mon[32712]: pgmap v505: 97 pgs: 97 active+clean; 298 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-04-25T16:54:17.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:17 smithi116 ceph-mon[32712]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T16:54:18.136 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 305, 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 295, 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 1564208, 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 181680, 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 0, 2024-04-25T16:54:18.137 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.051725859, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.015838543, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.039087965, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.005650696, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 294, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.010730983, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000036499 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.138 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 1564208, 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 181680 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:18.139 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 3, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 226, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 480, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 275840, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 6, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 3, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.015572831, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.010626886, 2024-04-25T16:54:18.140 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.002418912, 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 223, 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.009827963, 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000044071 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:18.141 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 275840 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_messages": 4, 2024-04-25T16:54:18.142 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages": 4, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_bytes": 3040, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_bytes": 448, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_created_connections": 2, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_total_time": 0.000639071, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_send_time": 0.000030533, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_recv_time": 0.000572032, 2024-04-25T16:54:18.143 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 3, 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000779227, 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000259742 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:18.144 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T16:54:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T16:54:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.145 INFO:teuthology.orchestra.run.smithi112.stdout: "cct": { 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "total_workers": 0, 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "unhealthy_workers": 0 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "mempool": { 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:18.146 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:18.147 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:18.148 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:18.149 INFO:teuthology.orchestra.run.smithi112.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_items": 0, 2024-04-25T16:54:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:18.150 INFO:teuthology.orchestra.run.smithi112.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_bytes": 148714, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_anon_items": 30, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_bytes": 0, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_items": 0, 2024-04-25T16:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_bytes": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_items": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:18.152 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "pgmap_items": 0, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_bytes": 928, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_co_items": 2, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_1_items": 0, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:18.153 INFO:teuthology.orchestra.run.smithi112.stdout: "unittest_2_items": 0 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "val": 0, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "max": 104857600, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "get_started": 0, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "get": 308, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "get_sum": 1521140, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:18.154 INFO:teuthology.orchestra.run.smithi112.stdout: "get_or_fail_success": 308, 2024-04-25T16:54:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "take": 0, 2024-04-25T16:54:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "take_sum": 0, 2024-04-25T16:54:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "put": 308, 2024-04-25T16:54:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "put_sum": 1521140, 2024-04-25T16:54:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "wait": { 2024-04-25T16:54:18.155 INFO:teuthology.orchestra.run.smithi112.stdout: "avgcount": 0, 2024-04-25T16:54:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: "sum": 0.000000000, 2024-04-25T16:54:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: "avgtime": 0.000000000 2024-04-25T16:54:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:18.156 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T16:54:18.156 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T16:54:18.840 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T16:54:18.841 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T16:54:18.841 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T16:54:18.841 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T16:54:18.841 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T16:54:18.841 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T16:54:18.841 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T16:54:18.841 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T16:54:18.842 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T16:54:19.140 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:54:19.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:19 smithi112 ceph-mon[29122]: pgmap v507: 97 pgs: 97 active+clean; 298 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-04-25T16:54:19.623 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:19 smithi116 ceph-mon[32712]: pgmap v507: 97 pgs: 97 active+clean; 298 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-04-25T16:54:19.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:19 smithi133 ceph-mon[32691]: pgmap v507: 97 pgs: 97 active+clean; 298 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.8 MiB/s wr, 118 op/s 2024-04-25T16:54:21.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:21 smithi112 ceph-mon[29122]: pgmap v508: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T16:54:21.679 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:21 smithi116 ceph-mon[32712]: pgmap v508: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T16:54:21.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:21 smithi133 ceph-mon[32691]: pgmap v508: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T16:54:21.780 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-04-25T16:54:21.780 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 286892, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 487025, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 57119910, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 774470039, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 15, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 4, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 25.337670255, 2024-04-25T16:54:21.781 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 12.538149565, 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 10.857188885, 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 2.534462995, 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 487026, 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 45.239096389, 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000092888 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.782 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 607832, 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.117246828, 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000192 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 533040, 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 156592 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:21.783 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 687357, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 965364, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 238891466, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 1196374325, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 148, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 9, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 42.545020846, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 19.109769552, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 27.351108206, 2024-04-25T16:54:21.784 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 2.547664478, 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 965361, 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 78.683872688, 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000081507 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1428415, 2024-04-25T16:54:21.785 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.202329143, 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000141 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 216335600, 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 830065168 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 247739, 2024-04-25T16:54:21.786 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 448568, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 46706647, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 416057018, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 9, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 7, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 18.244930300, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 8.149372626, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 8.844138209, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 1.443660047, 2024-04-25T16:54:21.787 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 448565, 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 37.721769395, 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000084094 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 603652, 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.112080795, 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000185 2024-04-25T16:54:21.788 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 1736336, 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 196832 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 1, 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.789 INFO:teuthology.orchestra.run.smithi116.stdout: "finisher-MDSRank": { 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: "queue_len": 0, 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: "complete_latency": { 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 172517, 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 213.194397793, 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.001235787 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.790 INFO:teuthology.orchestra.run.smithi116.stdout: "finisher-PurgeQueue": { 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "queue_len": 0, 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "complete_latency": { 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 97835, 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 19.294994256, 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000197219 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": { 2024-04-25T16:54:21.791 INFO:teuthology.orchestra.run.smithi116.stdout: "request": 475556, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "reply": 475179, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "reply_latency": { 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 475179, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 1623.447518692, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003416496 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "slow_reply": 0, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "forward": 377, 2024-04-25T16:54:21.792 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_fetch_complete": 0, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_fetch_keys": 0, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_commit": 5347, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_split": 30, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_merge": 34, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes": 6267, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_top": 2100, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_bottom": 4166, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_pin_tail": 1, 2024-04-25T16:54:21.793 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_pinned": 3263, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_expired": 106, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_with_caps": 1226, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": 1227, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "subtrees": 9, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse": 694345, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_hit": 429913, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_forward": 376, 2024-04-25T16:54:21.794 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_discover": 1, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_dir_fetch": 0, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_remote_ino": 0, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_lock": 4545, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "load_cent": 0, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "q": 0, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "exported": 5, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "exported_inodes": 64, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "imported": 0, 2024-04-25T16:54:21.795 INFO:teuthology.orchestra.run.smithi116.stdout: "imported_inodes": 0, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "openino_dir_fetch": 0, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "openino_backtrace_fetch": 0, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "openino_peer_discover": 0, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "root_rfiles": 702, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "root_rbytes": 211896570, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "root_rsnaps": 6, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_set_tag": 0, 2024-04-25T16:54:21.796 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_inotable_repaired": 0, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_dir_inodes": 0, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_file_inodes": 0, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_inode_file_caps": 5, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_revoke": 18626, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_grant": 216141, 2024-04-25T16:54:21.797 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_caps": 110219, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_caps_dirty": 49763, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_cap_release": 23201, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "process_request_cap_release": 331401 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_cache": { 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_update": 204, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_update_receipt": 0, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_try_discover": 0, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_send_discover": 1, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_handle_discover": 497, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "num_strays": 580, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "num_strays_delayed": 0, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "num_strays_enqueuing": 0, 2024-04-25T16:54:21.799 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_created": 61348, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_enqueued": 60768, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_reintegrated": 0, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_migrated": 0, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "num_recovering_processing": 0, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "num_recovering_enqueued": 0, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "num_recovering_prioritized": 0, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_started": 0, 2024-04-25T16:54:21.800 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_completed": 0, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_quiesce_path": 0, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_quiesce_inode": 0, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_exportdir": 5, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_flush": 0, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_fragmentdir": 64, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_fragstats": 0, 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_inodestats": 0 2024-04-25T16:54:21.801 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_log": { 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "evlrg": 0, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "evadd": 290964, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "evex": 284162, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "evtrm": 284162, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "ev": 6802, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "evexg": 0, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "evexd": 6830, 2024-04-25T16:54:21.802 INFO:teuthology.orchestra.run.smithi116.stdout: "segadd": 450, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "segex": 440, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "segtrm": 440, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "seg": 10, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "segmjr": 1, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "segexg": 0, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "segexd": 11, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "expos": 1047992499, 2024-04-25T16:54:21.803 INFO:teuthology.orchestra.run.smithi116.stdout: "wrpos": 1067201589, 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "rdpos": 0, 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "jlat": { 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 90369, 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 429.400393436, 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.004751633 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "replayed": 0 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.804 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_mem": { 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 3335, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "ino+": 64134, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "ino-": 60799, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "dir": 201, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "dir+": 407, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "dir-": 206, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "dn": 6274, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "dn+": 142495, 2024-04-25T16:54:21.805 INFO:teuthology.orchestra.run.smithi116.stdout: "dn-": 136221, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "cap": 968, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "cap+": 68179, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "cap-": 67211, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "rss": 185836, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "heap": 231824 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_server": { 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "dispatch_client_request": 698782, 2024-04-25T16:54:21.806 INFO:teuthology.orchestra.run.smithi116.stdout: "dispatch_server_request": 1493, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_request": 475556, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_session": 8630, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_peer_request": 2983, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "req_create_latency": { 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 62050, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 225.886815334, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003640399 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.807 INFO:teuthology.orchestra.run.smithi116.stdout: "req_getattr_latency": { 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1792, 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 44.026091117, 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.024568131 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "req_getfilelock_latency": { 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.808 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "req_link_latency": { 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookup_latency": { 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 146116, 2024-04-25T16:54:21.809 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 375.796001647, 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.002571901 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookuphash_latency": { 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupino_latency": { 2024-04-25T16:54:21.810 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1, 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000375589, 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000375589 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupname_latency": { 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.811 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupparent_latency": { 2024-04-25T16:54:21.812 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.812 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.812 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.812 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.813 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupsnap_latency": { 2024-04-25T16:54:21.813 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.813 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.813 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.813 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.813 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lssnap_latency": { 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 16, 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.010041027, 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000627564 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "req_mkdir_latency": { 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 119, 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 1.096685248, 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.009215842 2024-04-25T16:54:21.814 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "req_mknod_latency": { 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "req_mksnap_latency": { 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 16, 2024-04-25T16:54:21.815 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 1.358357397, 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.084897337 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "req_open_latency": { 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 10, 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.037353831, 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003735383 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "req_readdir_latency": { 2024-04-25T16:54:21.816 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 188893, 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 694.106730958, 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003674602 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rename_latency": { 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 12751, 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 49.900959726, 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003913493 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.817 INFO:teuthology.orchestra.run.smithi116.stdout: "req_renamesnap_latency": { 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "req_snapdiff_latency": { 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.818 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rmdir_latency": { 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1, 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.003763105, 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003763105 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rmsnap_latency": { 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 10, 2024-04-25T16:54:21.819 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 2.286176826, 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.228617682 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rmxattr_latency": { 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setattr_latency": { 2024-04-25T16:54:21.820 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 56, 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 2.819560705, 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.050349298 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setdirlayout_latency": { 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.821 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setfilelock_latency": { 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1992, 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 5.066890645, 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.002543619 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setlayout_latency": { 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.822 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setxattr_latency": { 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 8, 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.021462657, 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.002682832 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "req_symlink_latency": { 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.823 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: "req_unlink_latency": { 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 61348, 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 221.030252880, 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.003602892 2024-04-25T16:54:21.824 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: "cap_revoke_eviction": 0, 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: "cap_acquisition_throttle": 0, 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: "req_getvxattr_latency": { 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.825 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_sessions": { 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "session_count": 2, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "session_add": 6, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "session_remove": 4, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "sessions_open": 2, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "sessions_stale": 0, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "total_load": 54402, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "average_load": 27201, 2024-04-25T16:54:21.826 INFO:teuthology.orchestra.run.smithi116.stdout: "avg_session_uptime": 938, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "mdthresh_evicted": 0 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:21.827 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:21.828 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:21.829 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:21.830 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 1318596, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 2327, 2024-04-25T16:54:21.831 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 8320, 2024-04-25T16:54:21.832 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 173, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 13623170, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 202122, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-04-25T16:54:21.833 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "objecter": { 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "op_active": 0, 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "op_laggy": 0, 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "op_send": 193221, 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "op_send_bytes": 1115484407, 2024-04-25T16:54:21.834 INFO:teuthology.orchestra.run.smithi116.stdout: "op_resend": 37, 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "op_reply": 193221, 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "op_latency": { 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 193221, 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 4533.912107694, 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.023464903 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "op_inflight": 0, 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "oplen_avg": { 2024-04-25T16:54:21.835 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 193221, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 268692 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "op": 193221, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "op_r": 1551, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "op_w": 191670, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "op_rmw": 0, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "op_pg": 0, 2024-04-25T16:54:21.836 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_stat": 4947, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_create": 30427, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_read": 1551, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_write": 91920, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_writefull": 986, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_writesame": 0, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_append": 0, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_zero": 0, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_truncate": 30, 2024-04-25T16:54:21.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_delete": 61959, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_mapext": 0, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_sparse_read": 0, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_clonerange": 0, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_getxattr": 0, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_setxattr": 60854, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_cmpxattr": 0, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_rmxattr": 0, 2024-04-25T16:54:21.838 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_resetxattrs": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_call": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_watch": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_notify": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_pgls": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_pgls_filter": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_other": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_active": 0, 2024-04-25T16:54:21.839 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_send": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_resend": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_ping": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "poolop_active": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "poolop_send": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "poolop_resend": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "poolstat_active": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "poolstat_send": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "poolstat_resend": 0, 2024-04-25T16:54:21.840 INFO:teuthology.orchestra.run.smithi116.stdout: "statfs_active": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "statfs_send": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "statfs_resend": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "command_active": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "command_send": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "command_resend": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "map_epoch": 109, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "map_full": 0, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "map_inc": 35, 2024-04-25T16:54:21.841 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_sessions": 12, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_session_open": 12, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_session_close": 0, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_laggy": 0, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_wr": 11681, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_rd": 0, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_del": 4337 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "oft": { 2024-04-25T16:54:21.842 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_objs": 1, 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_kv_pairs": 0, 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_updates": 0, 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_removes": 0 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "osd-slow-ops": { 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "slow_ops_count": 1 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "purge_queue": { 2024-04-25T16:54:21.843 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing_ops": 0, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing": 0, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing_high_water": 10, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executed_ops": 60768, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executed": 60768, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_item_in_journal": 0 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.844 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 1221984, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 204476652, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 1221984, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:21.845 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 1221984, 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 204476652, 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.846 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-objecter_bytes": { 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 0, 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 0, 2024-04-25T16:54:21.847 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 193221, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 1121630087, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 193221, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 1121630087, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.848 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-objecter_ops": { 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 1024, 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 0, 2024-04-25T16:54:21.849 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 0, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 193221, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 193221, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 193221, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 193221, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.850 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-write_buf_throttle": { 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 3758096384, 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 60768, 2024-04-25T16:54:21.851 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 6145680, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 60768, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 1551, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 6145680, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.852 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-write_buf_throttle-0x5618b78368f0": { 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 837788, 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 3758096384, 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 290964, 2024-04-25T16:54:21.853 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 1063007285, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 290964, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 90369, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 1062169497, 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:21.854 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:21.855 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:21.855 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:21.855 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.855 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:21.855 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds.exported=5 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=204 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-04-25T16:54:23.215 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-04-25T16:54:23.215 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T16:54:23.216 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T16:54:23.470 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:54:23.475 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:23 smithi116 ceph-mon[32712]: pgmap v509: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T16:54:23.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:23 smithi133 ceph-mon[32691]: pgmap v509: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T16:54:23.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:23 smithi112 ceph-mon[29122]: pgmap v509: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 2.9 MiB/s wr, 115 op/s 2024-04-25T16:54:25.593 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-04-25T16:54:25.594 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:25.594 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 0, 2024-04-25T16:54:25.595 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 2, 2024-04-25T16:54:25.595 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 0, 2024-04-25T16:54:25.595 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 0, 2024-04-25T16:54:25.595 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:25.596 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 0, 2024-04-25T16:54:25.596 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.001319652, 2024-04-25T16:54:25.596 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T16:54:25.596 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.000517776, 2024-04-25T16:54:25.597 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:25.597 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:25.597 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:25.597 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:25.598 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:25.598 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.598 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:25.598 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:25.599 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:25.599 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:25.599 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.600 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T16:54:25.600 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T16:54:25.600 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.600 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:25.600 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 133, 2024-04-25T16:54:25.601 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 130, 2024-04-25T16:54:25.601 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 189760, 2024-04-25T16:54:25.601 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 78576, 2024-04-25T16:54:25.601 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 2, 2024-04-25T16:54:25.602 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:25.602 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.013581528, 2024-04-25T16:54:25.602 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.003855580, 2024-04-25T16:54:25.602 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.007663580, 2024-04-25T16:54:25.603 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.002410143, 2024-04-25T16:54:25.603 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:25.603 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 130, 2024-04-25T16:54:25.603 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.005608847, 2024-04-25T16:54:25.603 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000043144 2024-04-25T16:54:25.604 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.604 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:25.604 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:25.604 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:25.604 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:25.605 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.605 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 189760, 2024-04-25T16:54:25.605 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 78576 2024-04-25T16:54:25.605 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.606 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:25.606 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 1, 2024-04-25T16:54:25.606 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 97, 2024-04-25T16:54:25.606 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 160, 2024-04-25T16:54:25.607 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 95584, 2024-04-25T16:54:25.607 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 3, 2024-04-25T16:54:25.607 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 2, 2024-04-25T16:54:25.607 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.005458729, 2024-04-25T16:54:25.607 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.002948580, 2024-04-25T16:54:25.608 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.001782278, 2024-04-25T16:54:25.608 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:25.608 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:25.608 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 95, 2024-04-25T16:54:25.608 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.002845005, 2024-04-25T16:54:25.609 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000029947 2024-04-25T16:54:25.609 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.609 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:25.609 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:25.610 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:25.610 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:25.610 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.610 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T16:54:25.610 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 95584 2024-04-25T16:54:25.611 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.611 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-04-25T16:54:25.611 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 0, 2024-04-25T16:54:25.611 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-04-25T16:54:25.611 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.612 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-04-25T16:54:25.612 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:25.612 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:25.612 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:25.613 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:25.613 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:25.613 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:25.613 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:25.614 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:25.614 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:25.614 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:25.615 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:25.615 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:25.615 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:25.615 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:25.616 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:25.616 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:25.616 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:25.616 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:25.616 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:25.617 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:25.617 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:25.617 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:25.617 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:25.617 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:25.618 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 149854, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 28, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:25.619 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 0, 2024-04-25T16:54:25.620 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 928, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 2, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:25.621 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 130, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 168496, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:25.622 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 130, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 130, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 168496, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:25.623 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:25.624 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:25.624 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-04-25T16:54:25.624 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:25 smithi116 ceph-mon[32712]: pgmap v510: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 3.1 MiB/s wr, 110 op/s 2024-04-25T16:54:25.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:25 smithi133 ceph-mon[32691]: pgmap v510: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 3.1 MiB/s wr, 110 op/s 2024-04-25T16:54:25.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:25 smithi112 ceph-mon[29122]: pgmap v510: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 3.1 MiB/s wr, 110 op/s 2024-04-25T16:54:26.443 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-25T16:54:26.443 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-25T16:54:26.443 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-04-25T16:54:26.444 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-04-25T16:54:26.444 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T16:54:26.444 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T16:54:26.444 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T16:54:26.444 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T16:54:26.444 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T16:54:26.728 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:54:27.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:27 smithi116 ceph-mon[32712]: pgmap v511: 97 pgs: 97 active+clean; 305 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.4 MiB/s wr, 92 op/s 2024-04-25T16:54:27.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:27 smithi133 ceph-mon[32691]: pgmap v511: 97 pgs: 97 active+clean; 305 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.4 MiB/s wr, 92 op/s 2024-04-25T16:54:27.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:27 smithi112 ceph-mon[29122]: pgmap v511: 97 pgs: 97 active+clean; 305 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 3.4 MiB/s wr, 92 op/s 2024-04-25T16:54:28.523 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-04-25T16:54:28.523 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:28.523 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 449341, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 249605, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 201816711, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 45248973, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 11, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 5, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 20.057014122, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 8.436745342, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 16.166095064, 2024-04-25T16:54:28.524 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.036262755, 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 249603, 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 3.370631925, 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000013503 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 696098, 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.096684652, 2024-04-25T16:54:28.525 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000138 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 13693728, 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 8807920 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 525, 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 683, 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 1720126, 2024-04-25T16:54:28.526 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 715709, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 14, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 5, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.069205920, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.021503370, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.050643496, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.009475626, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 683, 2024-04-25T16:54:28.527 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.054190392, 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000079341 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 15, 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000002010, 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000134 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 1689088, 2024-04-25T16:54:28.528 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 626768 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 406, 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 464, 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 132905, 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 343487, 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 13, 2024-04-25T16:54:28.529 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 6, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.038900808, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.010045600, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.015886804, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.010223382, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 461, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.033260332, 2024-04-25T16:54:28.530 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000072148 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 64, 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000012612, 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000197 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 4224, 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 142256 2024-04-25T16:54:28.531 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 1, 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "finisher-MDSRank": { 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "queue_len": 0, 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "complete_latency": { 2024-04-25T16:54:28.532 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1503, 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.762014455, 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000506995 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "finisher-PurgeQueue": { 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "queue_len": 0, 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "complete_latency": { 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 639, 2024-04-25T16:54:28.533 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.035527188, 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000055598 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": { 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: "request": 42218, 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: "reply": 1630, 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: "reply_latency": { 2024-04-25T16:54:28.534 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1630, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 12.985508257, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.007966569 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "slow_reply": 0, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "forward": 40588, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_fetch_complete": 0, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_fetch_keys": 0, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_commit": 11, 2024-04-25T16:54:28.535 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_split": 0, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_merge": 0, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes": 208, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_top": 62, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_bottom": 146, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_pin_tail": 0, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_pinned": 118, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_expired": 0, 2024-04-25T16:54:28.536 INFO:teuthology.orchestra.run.smithi116.stdout: "inodes_with_caps": 24, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": 24, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "subtrees": 4, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse": 45032, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_hit": 2150, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_forward": 40587, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_discover": 107, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_dir_fetch": 0, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_remote_ino": 0, 2024-04-25T16:54:28.537 INFO:teuthology.orchestra.run.smithi116.stdout: "traverse_lock": 99, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "load_cent": 0, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "q": 0, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "exported": 0, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "exported_inodes": 0, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "imported": 2, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "imported_inodes": 25, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "openino_dir_fetch": 0, 2024-04-25T16:54:28.538 INFO:teuthology.orchestra.run.smithi116.stdout: "openino_backtrace_fetch": 0, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "openino_peer_discover": 0, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "root_rfiles": 702, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "root_rbytes": 211886132, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "root_rsnaps": 6, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_set_tag": 0, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_inotable_repaired": 0, 2024-04-25T16:54:28.539 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_dir_inodes": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "scrub_file_inodes": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_inode_file_caps": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_revoke": 41, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_grant": 1103, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T16:54:28.540 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_caps": 781, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_caps_dirty": 513, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_cap_release": 100, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "process_request_cap_release": 1547 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_cache": { 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_update": 35, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_update_receipt": 398, 2024-04-25T16:54:28.541 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_try_discover": 250, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_send_discover": 306, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "dir_handle_discover": 37, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "num_strays": 8, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "num_strays_delayed": 0, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "num_strays_enqueuing": 0, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_created": 278, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_enqueued": 270, 2024-04-25T16:54:28.542 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_reintegrated": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "strays_migrated": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "num_recovering_processing": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "num_recovering_enqueued": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "num_recovering_prioritized": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_started": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_completed": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_quiesce_path": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_quiesce_inode": 0, 2024-04-25T16:54:28.543 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_exportdir": 0, 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_flush": 0, 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_fragmentdir": 0, 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_fragstats": 0, 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "ireq_inodestats": 0 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_log": { 2024-04-25T16:54:28.544 INFO:teuthology.orchestra.run.smithi116.stdout: "evlrg": 0, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "evadd": 2054, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "evex": 0, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "evtrm": 0, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "ev": 2054, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "evexg": 0, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "evexd": 0, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "segadd": 4, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "segex": 0, 2024-04-25T16:54:28.545 INFO:teuthology.orchestra.run.smithi116.stdout: "segtrm": 0, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "seg": 4, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "segmjr": 2, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "segexg": 0, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "segexd": 0, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "expos": 4194304, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "wrpos": 7114276, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "rdpos": 0, 2024-04-25T16:54:28.546 INFO:teuthology.orchestra.run.smithi116.stdout: "jlat": { 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 710, 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 4.310920420, 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.006071718 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "replayed": 0 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_mem": { 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "ino": 193, 2024-04-25T16:54:28.547 INFO:teuthology.orchestra.run.smithi116.stdout: "ino+": 449, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "ino-": 256, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "dir": 147, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "dir+": 206, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "dir-": 59, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "dn": 208, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "dn+": 584, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "dn-": 376, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "cap": 24, 2024-04-25T16:54:28.548 INFO:teuthology.orchestra.run.smithi116.stdout: "cap+": 309, 2024-04-25T16:54:28.549 INFO:teuthology.orchestra.run.smithi116.stdout: "cap-": 285, 2024-04-25T16:54:28.549 INFO:teuthology.orchestra.run.smithi116.stdout: "rss": 130772, 2024-04-25T16:54:28.549 INFO:teuthology.orchestra.run.smithi116.stdout: "heap": 223632 2024-04-25T16:54:28.549 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.549 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_server": { 2024-04-25T16:54:28.549 INFO:teuthology.orchestra.run.smithi116.stdout: "dispatch_client_request": 43949, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "dispatch_server_request": 0, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_request": 42218, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_client_session": 133, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "handle_peer_request": 666, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "req_create_latency": { 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 278, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 4.941507998, 2024-04-25T16:54:28.550 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.017775208 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "req_getattr_latency": { 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 13, 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.009623980, 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000740306 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "req_getfilelock_latency": { 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.551 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "req_link_latency": { 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookup_latency": { 2024-04-25T16:54:28.552 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 542, 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 4.174826724, 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.007702632 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookuphash_latency": { 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.553 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupino_latency": { 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupname_latency": { 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.554 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupparent_latency": { 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lookupsnap_latency": { 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.555 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.556 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.556 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.556 INFO:teuthology.orchestra.run.smithi116.stdout: "req_lssnap_latency": { 2024-04-25T16:54:28.556 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: "req_mkdir_latency": { 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.557 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: "req_mknod_latency": { 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: "req_mksnap_latency": { 2024-04-25T16:54:28.558 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "req_open_latency": { 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "req_readdir_latency": { 2024-04-25T16:54:28.559 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 415, 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.292298857, 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000704334 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rename_latency": { 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 99, 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 1.646286619, 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.016629157 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.560 INFO:teuthology.orchestra.run.smithi116.stdout: "req_renamesnap_latency": { 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "req_snapdiff_latency": { 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.561 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rmdir_latency": { 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rmsnap_latency": { 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.562 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "req_rmxattr_latency": { 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setattr_latency": { 2024-04-25T16:54:28.563 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 5, 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.026653213, 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.005330642 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setdirlayout_latency": { 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.564 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setfilelock_latency": { 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setlayout_latency": { 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.565 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: "req_setxattr_latency": { 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.566 INFO:teuthology.orchestra.run.smithi116.stdout: "req_symlink_latency": { 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "req_unlink_latency": { 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 278, 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 1.894310866, 2024-04-25T16:54:28.567 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.006814067 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: "cap_revoke_eviction": 0, 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: "cap_acquisition_throttle": 0, 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: "req_getvxattr_latency": { 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.568 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_sessions": { 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "session_count": 2, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "session_add": 4, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "session_remove": 2, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "sessions_open": 2, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "sessions_stale": 0, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "total_load": 958, 2024-04-25T16:54:28.569 INFO:teuthology.orchestra.run.smithi116.stdout: "average_load": 479, 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "avg_session_uptime": 896, 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "mdthresh_evicted": 0 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:28.570 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:28.571 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:28.572 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:28.573 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:28.573 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:28.574 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:28.574 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:28.574 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:28.574 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:28.574 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:28.574 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:28.575 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:28.576 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:28.576 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 499573, 2024-04-25T16:54:28.576 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 336, 2024-04-25T16:54:28.576 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:28.576 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:28.576 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 8320, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 173, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:28.577 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:28.578 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-04-25T16:54:28.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 854222, 2024-04-25T16:54:28.578 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 10205, 2024-04-25T16:54:28.578 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:28.578 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-04-25T16:54:28.578 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "objecter": { 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "op_active": 0, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "op_laggy": 0, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "op_send": 1294, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "op_send_bytes": 2989228, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "op_resend": 28, 2024-04-25T16:54:28.579 INFO:teuthology.orchestra.run.smithi116.stdout: "op_reply": 1294, 2024-04-25T16:54:28.580 INFO:teuthology.orchestra.run.smithi116.stdout: "op_latency": { 2024-04-25T16:54:28.580 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1294, 2024-04-25T16:54:28.580 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 7.785874484, 2024-04-25T16:54:28.580 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.006016904 2024-04-25T16:54:28.580 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.580 INFO:teuthology.orchestra.run.smithi116.stdout: "op_inflight": 0, 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "oplen_avg": { 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 1294, 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 1432 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "op": 1294, 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "op_r": 70, 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "op_w": 1224, 2024-04-25T16:54:28.581 INFO:teuthology.orchestra.run.smithi116.stdout: "op_rmw": 0, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "op_pg": 0, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_stat": 0, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_create": 10, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_read": 70, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_write": 780, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_writefull": 58, 2024-04-25T16:54:28.582 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_writesame": 0, 2024-04-25T16:54:28.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_append": 0, 2024-04-25T16:54:28.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_zero": 0, 2024-04-25T16:54:28.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_truncate": 0, 2024-04-25T16:54:28.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_delete": 282, 2024-04-25T16:54:28.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_mapext": 0, 2024-04-25T16:54:28.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_sparse_read": 0, 2024-04-25T16:54:28.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_clonerange": 0, 2024-04-25T16:54:28.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_getxattr": 0, 2024-04-25T16:54:28.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_setxattr": 20, 2024-04-25T16:54:28.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_cmpxattr": 0, 2024-04-25T16:54:28.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_rmxattr": 0, 2024-04-25T16:54:28.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_resetxattrs": 0, 2024-04-25T16:54:28.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_call": 0, 2024-04-25T16:54:28.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_watch": 0, 2024-04-25T16:54:28.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_notify": 0, 2024-04-25T16:54:28.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T16:54:28.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_pgls": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_pgls_filter": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "osdop_other": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_active": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_send": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_resend": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "linger_ping": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "poolop_active": 0, 2024-04-25T16:54:28.586 INFO:teuthology.orchestra.run.smithi116.stdout: "poolop_send": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "poolop_resend": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "poolstat_active": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "poolstat_send": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "poolstat_resend": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "statfs_active": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "statfs_send": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "statfs_resend": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "command_active": 0, 2024-04-25T16:54:28.587 INFO:teuthology.orchestra.run.smithi116.stdout: "command_send": 0, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "command_resend": 0, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "map_epoch": 109, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "map_full": 0, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "map_inc": 34, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_sessions": 12, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_session_open": 16, 2024-04-25T16:54:28.588 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_session_close": 4, 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_laggy": 0, 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_wr": 160, 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_rd": 0, 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_del": 52 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "oft": { 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_objs": 1, 2024-04-25T16:54:28.589 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_kv_pairs": 0, 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_updates": 0, 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: "omap_total_removes": 0 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: "osd-slow-ops": { 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: "slow_ops_count": 0 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: "purge_queue": { 2024-04-25T16:54:28.590 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing_ops": 0, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing": 0, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executing_high_water": 10, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executed_ops": 270, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_executed": 270, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "pq_item_in_journal": 0 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:28.591 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 450268, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 161074591, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 450268, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:28.592 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 450268, 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 161074591, 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.593 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-objecter_bytes": { 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 0, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 0, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:28.594 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 1294, 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 3016586, 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 1294, 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 3016586, 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.595 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-objecter_ops": { 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 1024, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 0, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 0, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:28.596 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 1294, 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 1294, 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 1294, 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 1294, 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.597 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-write_buf_throttle": { 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 3758096384, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 270, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 27358, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:28.598 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 270, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 70, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 27358, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.599 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-write_buf_throttle-0x558291fb4680": { 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 8434, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 3758096384, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 2054, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 2919972, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:28.600 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 2054, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 710, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 2911538, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:28.601 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.602 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:28.602 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-04-25T16:54:29.537 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T16:54:29.537 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-04-25T16:54:29.537 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=35 2024-04-25T16:54:29.537 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=398 2024-04-25T16:54:29.537 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T16:54:29.538 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T16:54:29.538 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T16:54:29.538 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T16:54:29.538 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T16:54:29.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:29 smithi133 ceph-mon[32691]: pgmap v512: 97 pgs: 97 active+clean; 305 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.2 MiB/s wr, 87 op/s 2024-04-25T16:54:29.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:29 smithi116 ceph-mon[32712]: pgmap v512: 97 pgs: 97 active+clean; 305 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.2 MiB/s wr, 87 op/s 2024-04-25T16:54:29.780 INFO:teuthology.orchestra.run.smithi116.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T16:54:29.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:29 smithi112 ceph-mon[29122]: pgmap v512: 97 pgs: 97 active+clean; 305 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 3.2 MiB/s wr, 87 op/s 2024-04-25T16:54:31.507 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:31 smithi116 ceph-mon[32712]: pgmap v513: 97 pgs: 97 active+clean; 295 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 4.6 MiB/s wr, 132 op/s 2024-04-25T16:54:31.569 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-04-25T16:54:31.569 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:31.570 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 0, 2024-04-25T16:54:31.570 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 2, 2024-04-25T16:54:31.570 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 0, 2024-04-25T16:54:31.570 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 0, 2024-04-25T16:54:31.570 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:31.570 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 0, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.001571754, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.000436176, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:31.571 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T16:54:31.572 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 1, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 37, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 160, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 85024, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 3, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 2, 2024-04-25T16:54:31.573 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.003664411, 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.001260710, 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.002025853, 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 35, 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.002842320, 2024-04-25T16:54:31.574 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000081209 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 85024 2024-04-25T16:54:31.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_messages": 55, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages": 52, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_bytes": 167232, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_bytes": 29744, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_created_connections": 2, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:31.576 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_total_time": 0.006724117, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_send_time": 0.001645617, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_recv_time": 0.004861901, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_running_fast_dispatch_time": 0.000965734, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 52, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.003694061, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000071039 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.577 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_recv_encrypted_bytes": 167232, 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: "msgr_send_encrypted_bytes": 29744 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.578 INFO:teuthology.orchestra.run.smithi116.stdout: "cct": { 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "total_workers": 0, 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "unhealthy_workers": 0 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "mempool": { 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:31.579 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:31.580 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:31.581 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:31.582 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_items": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:31.583 INFO:teuthology.orchestra.run.smithi116.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_bytes": 149070, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_anon_items": 26, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_bytes": 0, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_items": 0, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:31.584 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_bytes": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_items": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:31.585 INFO:teuthology.orchestra.run.smithi116.stdout: "pgmap_items": 0, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_bytes": 928, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_co_items": 2, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_1_items": 0, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "unittest_2_items": 0 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-04-25T16:54:31.586 INFO:teuthology.orchestra.run.smithi116.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "val": 0, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "max": 104857600, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "get_started": 0, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "get": 52, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "get_sum": 157018, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "get_or_fail_success": 52, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "take": 0, 2024-04-25T16:54:31.587 INFO:teuthology.orchestra.run.smithi116.stdout: "take_sum": 0, 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: "put": 52, 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: "put_sum": 157018, 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: "wait": { 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: "avgcount": 0, 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: "sum": 0.000000000, 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: "avgtime": 0.000000000 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-04-25T16:54:31.588 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-04-25T16:54:31.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:31 smithi133 ceph-mon[32691]: pgmap v513: 97 pgs: 97 active+clean; 295 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 4.6 MiB/s wr, 132 op/s 2024-04-25T16:54:31.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:31 smithi112 ceph-mon[29122]: pgmap v513: 97 pgs: 97 active+clean; 295 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 4.6 MiB/s wr, 132 op/s 2024-04-25T16:54:32.326 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T16:54:32.326 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T16:54:32.330 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T16:54:32.330 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T16:54:32.330 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T16:54:32.330 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T16:54:32.330 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T16:54:32.330 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T16:54:32.331 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T16:54:32.575 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:54:32.697 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:32 smithi133 ceph-mon[32691]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:54:32.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:32 smithi116 ceph-mon[32712]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:54:32.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:32 smithi112 ceph-mon[29122]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T16:54:33.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:33 smithi133 ceph-mon[32691]: pgmap v514: 97 pgs: 97 active+clean; 295 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.9 MiB/s wr, 86 op/s 2024-04-25T16:54:33.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:33 smithi116 ceph-mon[32712]: pgmap v514: 97 pgs: 97 active+clean; 295 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.9 MiB/s wr, 86 op/s 2024-04-25T16:54:33.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:33 smithi112 ceph-mon[29122]: pgmap v514: 97 pgs: 97 active+clean; 295 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.9 MiB/s wr, 86 op/s 2024-04-25T16:54:34.459 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 3248, 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 3250, 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3837223, 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 1032871, 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 6, 2024-04-25T16:54:34.460 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.228836454, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.070902149, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.058278388, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.083249882, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3248, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.091290683, 2024-04-25T16:54:34.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000028106 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T16:54:34.462 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 9, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 231, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 4076, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 654058, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T16:54:34.463 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.011585285, 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.006902538, 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002760808, 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000077226, 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 229, 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.012830585, 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000056028 2024-04-25T16:54:34.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3520, 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 653360 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.465 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 366, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 337, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1676300, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 190094, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.056277446, 2024-04-25T16:54:34.466 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.016591910, 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.045068846, 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.006235371, 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 336, 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013294770, 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000039567 2024-04-25T16:54:34.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:34.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.468 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.475 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1676032, 2024-04-25T16:54:34.475 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 189776 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:34.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5470, 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.128013216, 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000023402 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:34.477 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.478 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.478 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.478 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.478 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T16:54:34.478 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.479 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 343, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-25T16:54:34.480 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 343, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 343, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 4, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-04-25T16:54:34.481 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T16:54:34.482 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-04-25T16:54:34.483 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T16:54:34.484 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T16:54:34.484 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T16:54:34.484 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T16:54:34.484 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-25T16:54:34.484 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T16:54:34.484 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T16:54:34.485 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T16:54:34.486 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-04-25T16:54:34.487 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-25T16:54:34.488 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T16:54:34.489 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 2055, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T16:54:34.490 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-04-25T16:54:34.491 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T16:54:34.491 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T16:54:34.491 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 4, 2024-04-25T16:54:34.491 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-04-25T16:54:34.491 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T16:54:34.492 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T16:54:34.492 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T16:54:34.492 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 7117310, 2024-04-25T16:54:34.492 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 7114276, 2024-04-25T16:54:34.492 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T16:54:34.493 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.493 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.493 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.494 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 2055 2024-04-25T16:54:34.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.494 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T16:54:34.494 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 66, 2024-04-25T16:54:34.495 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 336, 2024-04-25T16:54:34.495 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 270, 2024-04-25T16:54:34.495 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 17, 2024-04-25T16:54:34.495 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 17, 2024-04-25T16:54:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-04-25T16:54:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 343, 2024-04-25T16:54:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 343, 2024-04-25T16:54:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-04-25T16:54:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-04-25T16:54:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-04-25T16:54:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-04-25T16:54:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 121388, 2024-04-25T16:54:34.498 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215440 2024-04-25T16:54:34.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.498 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T16:54:34.498 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-04-25T16:54:34.499 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T16:54:34.499 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-04-25T16:54:34.499 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-04-25T16:54:34.499 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T16:54:34.500 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T16:54:34.500 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.500 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.500 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.501 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T16:54:34.501 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.501 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.502 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.502 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.502 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T16:54:34.502 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.503 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.503 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.503 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T16:54:34.504 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.504 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.504 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.505 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T16:54:34.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.505 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T16:54:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.506 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T16:54:34.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.508 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.508 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.508 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T16:54:34.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.509 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.510 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T16:54:34.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.510 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.511 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T16:54:34.511 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.511 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T16:54:34.512 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.513 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T16:54:34.515 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.515 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.515 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.516 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T16:54:34.516 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.516 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.516 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.517 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.517 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T16:54:34.517 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.517 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.518 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.518 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T16:54:34.518 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.519 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.519 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.519 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T16:54:34.520 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.520 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.520 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.521 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T16:54:34.521 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.521 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.521 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.522 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T16:54:34.522 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.522 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.523 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.523 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T16:54:34.523 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.524 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.524 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.524 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T16:54:34.525 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.525 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.525 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.526 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T16:54:34.526 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.526 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.527 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.527 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T16:54:34.527 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.529 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.529 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.529 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T16:54:34.530 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.530 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.530 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.531 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T16:54:34.531 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.531 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.531 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.532 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T16:54:34.532 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.532 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.533 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.533 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T16:54:34.533 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.534 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.534 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.534 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T16:54:34.535 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.535 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.535 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.536 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T16:54:34.536 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.536 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.536 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.537 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T16:54:34.537 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.537 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.538 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.538 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T16:54:34.538 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T16:54:34.539 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T16:54:34.539 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.539 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.539 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.540 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.540 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.540 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T16:54:34.541 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-25T16:54:34.541 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 4, 2024-04-25T16:54:34.541 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 2, 2024-04-25T16:54:34.541 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 3, 2024-04-25T16:54:34.541 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T16:54:34.542 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-04-25T16:54:34.542 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-04-25T16:54:34.542 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 2622, 2024-04-25T16:54:34.542 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T16:54:34.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.543 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T16:54:34.543 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:34.543 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:34.544 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:34.544 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:34.544 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:34.544 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:34.544 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:34.544 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:34.545 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:34.546 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:34.547 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 184233, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 106, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T16:54:34.548 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8320, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 173, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:34.549 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 427194, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 3740, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.550 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 3251, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 3251, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3251, 2024-04-25T16:54:34.551 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.712949106, 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000834496 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3251, 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3252 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 3251, 2024-04-25T16:54:34.552 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 3251, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2088, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1161, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-04-25T16:54:34.553 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T16:54:34.554 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T16:54:34.555 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T16:54:34.556 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T16:54:34.557 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 34, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 4, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 5, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 1, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T16:54:34.558 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T16:54:34.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T16:54:34.560 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:34.561 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 3619, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 5170252, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 3619, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3619, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 5170252, 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:34.562 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:34.563 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3251, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 2923011, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3251, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 2923011, 2024-04-25T16:54:34.564 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T16:54:34.565 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3251, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 3251, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3251, 2024-04-25T16:54:34.566 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3251, 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:34.567 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-25T16:54:34.568 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55caeea36c30": { 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:34.569 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:34.570 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:34.571 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T16:54:35.345 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T16:54:35.345 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T16:54:35.345 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T16:54:35.345 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T16:54:35.345 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-04-25T16:54:35.346 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T16:54:35.346 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T16:54:35.346 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T16:54:35.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T16:54:35.605 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:54:35.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:35 smithi133 ceph-mon[32691]: pgmap v515: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.2 MiB/s wr, 104 op/s 2024-04-25T16:54:35.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:35 smithi116 ceph-mon[32712]: pgmap v515: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.2 MiB/s wr, 104 op/s 2024-04-25T16:54:35.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:35 smithi112 ceph-mon[29122]: pgmap v515: 97 pgs: 97 active+clean; 296 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 3.2 MiB/s wr, 104 op/s 2024-04-25T16:54:37.428 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:37 smithi133 ceph-mon[32691]: pgmap v516: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 3.5 MiB/s wr, 121 op/s 2024-04-25T16:54:37.428 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T16:54:37.429 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:37.429 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 367, 2024-04-25T16:54:37.429 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 338, 2024-04-25T16:54:37.429 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1677138, 2024-04-25T16:54:37.429 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 190740, 2024-04-25T16:54:37.429 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 6, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.054992079, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.015880614, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.043569254, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.006418137, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 337, 2024-04-25T16:54:37.430 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013863866, 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000041139 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1676304, 2024-04-25T16:54:37.431 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 190416 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 974, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 1199, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 239454, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 873869, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 5, 2024-04-25T16:54:37.432 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.087547299, 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.027762591, 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.019662546, 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.031593602, 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1196, 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.043904877, 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000036709 2024-04-25T16:54:37.433 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 564768 2024-04-25T16:54:37.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 2244, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2244, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 4165071, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 712826, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.185359013, 2024-04-25T16:54:37.435 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.054603390, 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.055505302, 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.060116042, 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2243, 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.073364834, 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000032708 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:37.436 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T16:54:37.437 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 5350, 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.136184714, 2024-04-25T16:54:37.438 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000025455 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.439 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.440 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-04-25T16:54:37.441 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 417, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 0, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 417, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 417, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T16:54:37.442 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 5, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T16:54:37.443 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T16:54:37.444 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T16:54:37.445 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T16:54:37.446 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T16:54:37.447 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-04-25T16:54:37.448 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-25T16:54:37.449 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T16:54:37.450 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 2417, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-04-25T16:54:37.451 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 4, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 7690292, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 7688541, 2024-04-25T16:54:37.452 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 2417 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 88, 2024-04-25T16:54:37.453 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 398, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 310, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 21, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 21, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 0, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 417, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 417, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 0, 2024-04-25T16:54:37.454 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 122420, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215440 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T16:54:37.455 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.456 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.458 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T16:54:37.459 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.460 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.462 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T16:54:37.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.462 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.462 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.462 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.462 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T16:54:37.463 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.463 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.463 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.463 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.464 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T16:54:37.465 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.466 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T16:54:37.467 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T16:54:37.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.470 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T16:54:37.471 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T16:54:37.473 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.474 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 3, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 1, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 3, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-04-25T16:54:37.476 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 2625, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:37.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:37.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:37.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:37.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 193100, 2024-04-25T16:54:37.481 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 132, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:37.482 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8320, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 173, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 537058, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 4518, 2024-04-25T16:54:37.483 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 3213, 2024-04-25T16:54:37.484 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 21, 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 3, 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 3213, 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3213, 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.203958638, 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000685950 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.485 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3213, 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 3214 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 3213, 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 3213, 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-04-25T16:54:37.486 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 2054, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 1157, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T16:54:37.487 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 1, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-04-25T16:54:37.488 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T16:54:37.489 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T16:54:37.490 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T16:54:37.491 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 34, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 5, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 6, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 1, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-04-25T16:54:37.492 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.493 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-04-25T16:54:37.494 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 3581, 2024-04-25T16:54:37.495 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 5737727, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 3581, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3581, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 5737727, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.496 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:37.497 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3213, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 3495993, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3213, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3495993, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.498 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:37.499 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 3213, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 3213, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 3213, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3213, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.500 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:37.501 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:37.502 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x56241e92ac30": { 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:37.503 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-25T16:54:37.504 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:37.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:37.505 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:37.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:37.505 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.505 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:37.505 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T16:54:37.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:37 smithi116 ceph-mon[32712]: pgmap v516: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 3.5 MiB/s wr, 121 op/s 2024-04-25T16:54:37.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:37 smithi112 ceph-mon[29122]: pgmap v516: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 3.5 MiB/s wr, 121 op/s 2024-04-25T16:54:38.251 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T16:54:38.251 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T16:54:38.251 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T16:54:38.251 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T16:54:38.251 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T16:54:38.252 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T16:54:38.252 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T16:54:38.252 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T16:54:38.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T16:54:38.496 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:54:39.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:39 smithi133 ceph-mon[32691]: pgmap v517: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 2.7 MiB/s wr, 103 op/s 2024-04-25T16:54:39.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:39 smithi116 ceph-mon[32712]: pgmap v517: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 2.7 MiB/s wr, 103 op/s 2024-04-25T16:54:39.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:39 smithi112 ceph-mon[29122]: pgmap v517: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 2.7 MiB/s wr, 103 op/s 2024-04-25T16:54:40.579 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T16:54:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 44626, 2024-04-25T16:54:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 46396, 2024-04-25T16:54:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 14064394, 2024-04-25T16:54:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 13336132, 2024-04-25T16:54:40.580 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 15, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 3.454723454, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 1.313439445, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 2.977235463, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.028730154, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 46394, 2024-04-25T16:54:40.581 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.874574960, 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000018851 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 88550, 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013689276, 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000154 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.582 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 13840944, 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 9492960 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 706, 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 677, 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 1769016, 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 353448, 2024-04-25T16:54:40.583 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 10, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 7, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.077180715, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.022491775, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.052225840, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.014015094, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 676, 2024-04-25T16:54:40.584 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.036158732, 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000053489 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 17, 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000002310, 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000135 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1688208, 2024-04-25T16:54:40.585 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 198384 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 405600, 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 204863, 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 188062465, 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 34014168, 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 12, 2024-04-25T16:54:40.586 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 7, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 17.982761920, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 7.022568730, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 15.526962352, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.010636220, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 204861, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.929259207, 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000014298 2024-04-25T16:54:40.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 608751, 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.084951839, 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000139 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 7136, 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 584928 2024-04-25T16:54:40.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:40.589 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1635, 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.838664069, 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000512944 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:40.590 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 705, 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.034788308, 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000049345 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 42533, 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 1945, 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T16:54:40.591 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1945, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 11.596743573, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005962336 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 40588, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T16:54:40.592 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 11, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 229, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 70, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 159, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 128, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T16:54:40.593 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 9, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 9, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 5, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 46212, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 3071, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 40585, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 103, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T16:54:40.594 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 170, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 3, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 39, 2024-04-25T16:54:40.595 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 702, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 211935247, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T16:54:40.596 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 27, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 1363, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T16:54:40.597 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 768, 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 480, 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 153, 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 2006 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T16:54:40.598 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 46, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 376, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 239, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 277, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 50, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T16:54:40.599 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 318, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 318, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T16:54:40.600 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T16:54:40.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 2429, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 2429, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T16:54:40.602 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 4, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 4, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 2, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 4194304, 2024-04-25T16:54:40.603 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 7699840, 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 0, 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 768, 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 4.433682284, 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005773023 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 0 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.604 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 189, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 507, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 318, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 148, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 191, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 43, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 221, 2024-04-25T16:54:40.605 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 649, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 428, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 9, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 358, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 349, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 131028, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 223632 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.606 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 44712, 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 42533, 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 88, 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 827, 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 318, 2024-04-25T16:54:40.607 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 5.418577352, 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.017039551 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 14, 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.017636702, 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.001259764 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T16:54:40.608 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 586, 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.199423631, 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002046798 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.610 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1, 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000159716, 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000159716 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T16:54:40.611 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.612 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T16:54:40.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.613 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.613 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.613 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T16:54:40.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.614 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.614 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.614 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.615 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.616 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 534, 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.431122559, 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000807345 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T16:54:40.617 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 172, 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.624047775, 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.015256091 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.619 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.620 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 2, 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.012351822, 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.006175911 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.621 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.623 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T16:54:40.624 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 318, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1.893424016, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005954163 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.625 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.626 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.626 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T16:54:40.626 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-25T16:54:40.626 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 3, 2024-04-25T16:54:40.626 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 1, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 2, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 1087, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 543, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 954, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.627 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:40.628 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:40.629 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:40.630 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:40.631 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:40.632 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:40.632 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 359319, 2024-04-25T16:54:40.632 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 293, 2024-04-25T16:54:40.632 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 264, 2024-04-25T16:54:40.632 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 3, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8320, 2024-04-25T16:54:40.633 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 173, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 837777, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 6743, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:40.634 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 1407, 2024-04-25T16:54:40.635 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 3593080, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 33, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 1407, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1407, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 7.892853312, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.005609703 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T16:54:40.636 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1407, 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 1543 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 1407, 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 70, 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 1337, 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T16:54:40.637 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 0, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 10, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 70, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 838, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 57, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T16:54:40.638 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 330, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 20, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T16:54:40.639 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T16:54:40.640 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T16:54:40.641 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-04-25T16:54:40.642 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 34, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 12, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 17, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 5, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 164, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 0, 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 54 2024-04-25T16:54:40.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 1, 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T16:54:40.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 10, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 318, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 318, 2024-04-25T16:54:40.645 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 450928, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 161223225, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:40.646 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 450928, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 450928, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 161223225, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.647 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:40.648 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:40.649 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:40.649 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 1407, 2024-04-25T16:54:40.649 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 3625262, 2024-04-25T16:54:40.649 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1407, 2024-04-25T16:54:40.649 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3625262, 2024-04-25T16:54:40.649 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:40.650 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 1407, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 1407, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1407, 2024-04-25T16:54:40.651 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 1407, 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:40.652 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 318, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 32182, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 318, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 70, 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 32182, 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:40.654 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55b87734c680": { 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 1722, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 2429, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 3505536, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 2429, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:40.655 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 768, 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 3503814, 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:40.656 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T16:54:41.605 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T16:54:41.606 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-04-25T16:54:41.606 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=46 2024-04-25T16:54:41.606 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=376 2024-04-25T16:54:41.606 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T16:54:41.606 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T16:54:41.606 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T16:54:41.606 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T16:54:41.606 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 shell --fsid a7905982-0320-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T16:54:41.646 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:41 smithi112 ceph-mon[29122]: pgmap v518: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-04-25T16:54:41.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:41 smithi133 ceph-mon[32691]: pgmap v518: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-04-25T16:54:41.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:41 smithi116 ceph-mon[32712]: pgmap v518: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 337 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-04-25T16:54:41.848 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T16:54:43.611 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:43 smithi133 ceph-mon[32691]: pgmap v519: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-25T16:54:43.640 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 412, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 414, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 67991027, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 131026, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 7, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 3, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.064238124, 2024-04-25T16:54:43.641 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.009087223, 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.040358469, 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.011296086, 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 412, 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013004905, 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000031565 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.642 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T16:54:43.643 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 607, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 593, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 85326024, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 277121, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 5, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 4, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.105677843, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.020489357, 2024-04-25T16:54:43.644 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.081480972, 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.012435145, 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 593, 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.021105673, 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000035591 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:43.645 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 1595792, 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 189296 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T16:54:43.646 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 230, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 460, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 54596907, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 338002, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 8, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 6, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.050717139, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.013426858, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.028106517, 2024-04-25T16:54:43.647 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.005420678, 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 457, 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.025038523, 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000054788 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.648 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 265488 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 1, 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.649 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-MDSRank": { 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 1389, 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.131431946, 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000094623 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.650 INFO:teuthology.orchestra.run.smithi133.stdout: "finisher-PurgeQueue": { 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "queue_len": 0, 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "complete_latency": { 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": { 2024-04-25T16:54:43.651 INFO:teuthology.orchestra.run.smithi133.stdout: "request": 0, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "reply": 0, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "reply_latency": { 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_reply": 0, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "forward": 0, 2024-04-25T16:54:43.652 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_complete": 0, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_fetch_keys": 0, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_commit": 0, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_split": 0, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_merge": 0, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes": 16878, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_top": 9136, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_bottom": 7742, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pin_tail": 0, 2024-04-25T16:54:43.653 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_pinned": 3827, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_expired": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "inodes_with_caps": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "subtrees": 7, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_hit": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_forward": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_discover": 0, 2024-04-25T16:54:43.654 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_dir_fetch": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_remote_ino": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "traverse_lock": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "load_cent": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "q": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "exported": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "exported_inodes": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "imported": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "imported_inodes": 0, 2024-04-25T16:54:43.655 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_dir_fetch": 0, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_backtrace_fetch": 0, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "openino_peer_discover": 0, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rfiles": 702, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rbytes": 216659524, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "root_rsnaps": 6, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_set_tag": 0, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T16:54:43.656 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_inotable_repaired": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_inodes": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_file_inodes": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_inode_file_caps": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_grant": 0, 2024-04-25T16:54:43.657 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps": 0, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_caps_dirty": 0, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_cap_release": 0, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "process_request_cap_release": 0 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.658 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_cache": { 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_update_receipt": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_try_discover": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_send_discover": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "dir_handle_discover": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_delayed": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "num_strays_enqueuing": 0, 2024-04-25T16:54:43.659 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_created": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_enqueued": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_reintegrated": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "strays_migrated": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_processing": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_enqueued": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "num_recovering_prioritized": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_started": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "recovery_completed": 0, 2024-04-25T16:54:43.660 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_path": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_quiesce_inode": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_exportdir": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_flush": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragmentdir": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_fragstats": 0, 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: "ireq_inodestats": 0 2024-04-25T16:54:43.661 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_log": { 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evlrg": 0, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evadd": 0, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evex": 0, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evtrm": 0, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "ev": 68860, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evexg": 0, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "evexd": 0, 2024-04-25T16:54:43.662 INFO:teuthology.orchestra.run.smithi133.stdout: "segadd": 0, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segex": 0, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segtrm": 0, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "seg": 12, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segmjr": 10, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segexg": 0, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "segexd": 0, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "expos": 1047992499, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "wrpos": 1070306498, 2024-04-25T16:54:43.663 INFO:teuthology.orchestra.run.smithi133.stdout: "rdpos": 1070301293, 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "jlat": { 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "replayed": 68860 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_mem": { 2024-04-25T16:54:43.664 INFO:teuthology.orchestra.run.smithi133.stdout: "ino": 2141, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "ino+": 16188, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "ino-": 14047, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dir": 204, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dir+": 256, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dir-": 52, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dn": 16878, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dn+": 16942, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "dn-": 64, 2024-04-25T16:54:43.665 INFO:teuthology.orchestra.run.smithi133.stdout: "cap": 0, 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "cap+": 0, 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "cap-": 0, 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "rss": 150944, 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "heap": 215440 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_server": { 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_client_request": 0, 2024-04-25T16:54:43.666 INFO:teuthology.orchestra.run.smithi133.stdout: "dispatch_server_request": 0, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_request": 0, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_client_session": 0, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "handle_peer_request": 0, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_create_latency": { 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.667 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getattr_latency": { 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getfilelock_latency": { 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.668 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: "req_link_latency": { 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookup_latency": { 2024-04-25T16:54:43.669 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookuphash_latency": { 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.670 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupino_latency": { 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupname_latency": { 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.671 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupparent_latency": { 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lookupsnap_latency": { 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.672 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: "req_lssnap_latency": { 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.673 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.674 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mkdir_latency": { 2024-04-25T16:54:43.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.674 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.674 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mknod_latency": { 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: "req_mksnap_latency": { 2024-04-25T16:54:43.675 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: "req_open_latency": { 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "req_readdir_latency": { 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rename_latency": { 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.677 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "req_renamesnap_latency": { 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "req_snapdiff_latency": { 2024-04-25T16:54:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmdir_latency": { 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.679 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmsnap_latency": { 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "req_rmxattr_latency": { 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.680 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.681 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setattr_latency": { 2024-04-25T16:54:43.681 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.681 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.681 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.682 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setdirlayout_latency": { 2024-04-25T16:54:43.682 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.682 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.682 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.682 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setfilelock_latency": { 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setlayout_latency": { 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.683 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "req_setxattr_latency": { 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "req_symlink_latency": { 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.684 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.685 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.685 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.685 INFO:teuthology.orchestra.run.smithi133.stdout: "req_unlink_latency": { 2024-04-25T16:54:43.685 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.685 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.685 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_revoke_eviction": 0, 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: "cap_acquisition_throttle": 0, 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: "req_getvxattr_latency": { 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.686 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_sessions": { 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "session_count": 2, 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "session_add": 2, 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "session_remove": 0, 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_open": 0, 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "sessions_stale": 0, 2024-04-25T16:54:43.687 INFO:teuthology.orchestra.run.smithi133.stdout: "total_load": 0, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "average_load": 0, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "avg_session_uptime": 2077, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mdthresh_evicted": 0 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T16:54:43.688 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2024-04-25T16:54:43.689 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T16:54:43.690 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2024-04-25T16:54:43.691 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 316232, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 2601, 2024-04-25T16:54:43.692 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2024-04-25T16:54:43.693 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 8320, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 173, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 17554835, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 127527, 2024-04-25T16:54:43.694 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "objecter": { 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "op_active": 0, 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "op_laggy": 0, 2024-04-25T16:54:43.695 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send": 916, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "op_send_bytes": 26, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "op_resend": 4, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "op_reply": 916, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "op_latency": { 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 916, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 2.421954895, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.002644055 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.696 INFO:teuthology.orchestra.run.smithi133.stdout: "op_inflight": 0, 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "oplen_avg": { 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 916, 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 917 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "op": 916, 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "op_r": 916, 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "op_w": 0, 2024-04-25T16:54:43.697 INFO:teuthology.orchestra.run.smithi133.stdout: "op_rmw": 0, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "op_pg": 0, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_stat": 538, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_create": 0, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_read": 375, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_write": 0, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writefull": 0, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_writesame": 0, 2024-04-25T16:54:43.698 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_append": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_zero": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_truncate": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_delete": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_mapext": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_sparse_read": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_clonerange": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_getxattr": 2, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_setxattr": 0, 2024-04-25T16:54:43.699 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_cmpxattr": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_rmxattr": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_resetxattrs": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_call": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_watch": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_notify": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_pgls_filter": 0, 2024-04-25T16:54:43.700 INFO:teuthology.orchestra.run.smithi133.stdout: "osdop_other": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_active": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_send": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_resend": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "linger_ping": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_active": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_send": 0, 2024-04-25T16:54:43.701 INFO:teuthology.orchestra.run.smithi133.stdout: "poolop_resend": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_active": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_send": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "poolstat_resend": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_active": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_send": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "statfs_resend": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "command_active": 0, 2024-04-25T16:54:43.702 INFO:teuthology.orchestra.run.smithi133.stdout: "command_send": 0, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "command_resend": 0, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "map_epoch": 109, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "map_full": 0, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "map_inc": 6, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_sessions": 9, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_open": 9, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_session_close": 0, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_laggy": 0, 2024-04-25T16:54:43.703 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_wr": 0, 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_rd": 2, 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_del": 0 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "oft": { 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_objs": 0, 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_kv_pairs": 0, 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_updates": 0, 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: "omap_total_removes": 0 2024-04-25T16:54:43.704 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "osd-slow-ops": { 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "slow_ops_count": 0 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "purge_queue": { 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops": 0, 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing": 0, 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executing_high_water": 0, 2024-04-25T16:54:43.705 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed_ops": 0, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_executed": 0, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "pq_item_in_journal": 0 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 1245, 2024-04-25T16:54:43.706 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 207782011, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 1245, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 1245, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 207782011, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.707 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_bytes": { 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:43.708 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 916, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 206063183, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 916, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 206063183, 2024-04-25T16:54:43.709 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-objecter_ops": { 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 1024, 2024-04-25T16:54:43.710 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 916, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 916, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 916, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 916, 2024-04-25T16:54:43.711 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle": { 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:43.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-25T16:54:43.713 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-write_buf_throttle-0x55cd046d4750": { 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2024-04-25T16:54:43.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 3758096384, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2024-04-25T16:54:43.715 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 0, 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 0, 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-04-25T16:54:43.716 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-04-25T16:54:43.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:43 smithi116 ceph-mon[32712]: pgmap v519: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-25T16:54:43.844 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:43 smithi112 ceph-mon[29122]: pgmap v519: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-25T16:54:44.548 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T16:54:44.549 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T16:54:44.561 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T16:54:44.561 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:54:44.561 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:54:44.583 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T16:54:44.583 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:54:44.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:54:44.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/2899566645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 msgr2=0x7fbdbc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/2899566645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc102f00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009900 tx=0x7fbda402a410 comp rx=0 tx=0).stop 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/2899566645 shutdown_connections 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/2899566645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc10d380 0x7fbdbc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/2899566645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc102f00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/2899566645 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdbc108a20 0x7fbdbc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/2899566645 >> 172.21.15.112:0/2899566645 conn(0x7fbdbc078110 msgr2=0x7fbdbc078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.803+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/2899566645 shutdown_connections 2024-04-25T16:54:44.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.804+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/2899566645 wait complete. 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.804+0000 7fbdc0a9f640 1 Processor -- start 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.804+0000 7fbdc0a9f640 1 -- start start 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc108a20 0x7fbdbc19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdbc10d380 0x7fbdbc19bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112be0 con 0x7fbdbc102af0 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112a60 con 0x7fbdbc10d380 2024-04-25T16:54:44.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112d60 con 0x7fbdbc108a20 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc19b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40458/0 (socket says 172.21.15.112:40458) 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 -- 172.21.15.112:0/3098221952 learned_addr learned my addr 172.21.15.112:0/3098221952 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdb3fff640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc108a20 0x7fbdbc19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdbad9d640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdbc10d380 0x7fbdbc19bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 -- 172.21.15.112:0/3098221952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc108a20 msgr2=0x7fbdbc19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc108a20 0x7fbdbc19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 -- 172.21.15.112:0/3098221952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdbc10d380 msgr2=0x7fbdbc19bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:44.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdbc10d380 0x7fbdbc19bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 -- 172.21.15.112:0/3098221952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc1a6880 con 0x7fbdbc102af0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdb3fff640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc108a20 0x7fbdbc19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdba59c640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc19b540 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e9c0 tx=0x7fbdac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac00cd80 con 0x7fbdbc102af0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdbc1a6b70 con 0x7fbdbc102af0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdac0054d0 con 0x7fbdbc102af0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac010430 con 0x7fbdbc102af0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.805+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdbc1a7090 con 0x7fbdbc102af0 2024-04-25T16:54:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.807+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdbc103cd0 con 0x7fbdbc102af0 2024-04-25T16:54:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.808+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fbdac004fb0 con 0x7fbdbc102af0 2024-04-25T16:54:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.809+0000 7fbdb37fe640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fbda80afc60 0x7fbda80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.809+0000 7fbdb3fff640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fbda80afc60 0x7fbda80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.809+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7fbdac0df740 con 0x7fbdbc102af0 2024-04-25T16:54:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.809+0000 7fbdb3fff640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fbda80afc60 0x7fbda80b2170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009230 tx=0x7fbda4000bc0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.813+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbdac053ff0 con 0x7fbdbc102af0 2024-04-25T16:54:44.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.990+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdbc06afa0 con 0x7fbdbc102af0 2024-04-25T16:54:44.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.991+0000 7fbdb37fe640 1 -- 172.21.15.112:0/3098221952 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+23347 (secure 0 0 0) 0x7fbdbc06afa0 con 0x7fbdbc102af0 2024-04-25T16:54:44.989 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:54:44.989 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":109,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:54:17.171776+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/3947590362":"2024-04-26T16:38:27.501574+0000","172.21.15.112:6800/3947590362":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/386718339":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/4216056022":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/795264311":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/2445069646":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:54:44.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fbda80afc60 msgr2=0x7fbda80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:44.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fbda80afc60 0x7fbda80b2170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009230 tx=0x7fbda4000bc0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 msgr2=0x7fbdbc19b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:44.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc19b540 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00e9c0 tx=0x7fbdac00ee80 comp rx=0 tx=0).stop 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 shutdown_connections 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fbda80afc60 0x7fbda80b2170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdbc108a20 0x7fbdbc19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdbc10d380 0x7fbdbc19bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 --2- 172.21.15.112:0/3098221952 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbdbc102af0 0x7fbdbc19b540 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.993+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 >> 172.21.15.112:0/3098221952 conn(0x7fbdbc078110 msgr2=0x7fbdbc1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.994+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 shutdown_connections 2024-04-25T16:54:44.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:44.994+0000 7fbdc0a9f640 1 -- 172.21.15.112:0/3098221952 wait complete. 2024-04-25T16:54:45.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T16:31:41.074159+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T16:36:24.204313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T16:36:25.525471+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:54:45.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:54:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 -- 172.21.15.112:0/4134855799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c10d3a0 msgr2=0x7f407c10f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 --2- 172.21.15.112:0/4134855799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c10d3a0 0x7f407c10f7c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4074009ca0 tx=0x7f407402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 -- 172.21.15.112:0/4134855799 shutdown_connections 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 --2- 172.21.15.112:0/4134855799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c10d3a0 0x7f407c10f7c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 --2- 172.21.15.112:0/4134855799 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f407c103280 0x7f407c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 --2- 172.21.15.112:0/4134855799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407c108a40 0x7f407c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 -- 172.21.15.112:0/4134855799 >> 172.21.15.112:0/4134855799 conn(0x7f407c077880 msgr2=0x7f407c077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 -- 172.21.15.112:0/4134855799 shutdown_connections 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.116+0000 7f40847b2640 1 -- 172.21.15.112:0/4134855799 wait complete. 2024-04-25T16:54:45.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 Processor -- start 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 -- start start 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 0x7f407c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f407c108a40 0x7f407c19acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407c10d3a0 0x7f407c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f407c112c20 con 0x7f407c103280 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f407c112aa0 con 0x7f407c108a40 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f40847b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f407c112da0 con 0x7f407c10d3a0 2024-04-25T16:54:45.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 0x7f407c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 0x7f407c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40470/0 (socket says 172.21.15.112:40470) 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 -- 172.21.15.112:0/4126900942 learned_addr learned my addr 172.21.15.112:0/4126900942 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f4082d4f640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407c10d3a0 0x7f407c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 -- 172.21.15.112:0/4126900942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407c10d3a0 msgr2=0x7f407c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407c10d3a0 0x7f407c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 -- 172.21.15.112:0/4126900942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f407c108a40 msgr2=0x7f407c19acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.117+0000 7f408254e640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f407c108a40 0x7f407c19acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f408254e640 1 -- 172.21.15.112:0/4126900942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c19f5b0 con 0x7f407c103280 2024-04-25T16:54:45.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f408254e640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 0x7f407c19a750 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d740 tx=0x7f406c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c017070 con 0x7f407c103280 2024-04-25T16:54:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c19f8a0 con 0x7f407c103280 2024-04-25T16:54:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f406c00de50 con 0x7f407c103280 2024-04-25T16:54:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c004ce0 con 0x7f407c103280 2024-04-25T16:54:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.118+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c1a7220 con 0x7f407c103280 2024-04-25T16:54:45.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.119+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c103d50 con 0x7f407c103280 2024-04-25T16:54:45.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.121+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f406c004e80 con 0x7f407c103280 2024-04-25T16:54:45.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.122+0000 7f40737fe640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40680afc60 0x7f40680b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.122+0000 7f4081d4d640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40680afc60 0x7f40680b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.122+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7f406c0deef0 con 0x7f407c103280 2024-04-25T16:54:45.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.122+0000 7f4081d4d640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40680afc60 0x7f40680b2170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f407c069e50 tx=0x7f4078005c10 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:45.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.125+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f406c0a7740 con 0x7f407c103280 2024-04-25T16:54:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.304+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f407c1a73f0 con 0x7f407c103280 2024-04-25T16:54:45.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.305+0000 7f40737fe640 1 -- 172.21.15.112:0/4126900942 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7f406c0a7040 con 0x7f407c103280 2024-04-25T16:54:45.303 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T16:54:45.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40680afc60 msgr2=0x7f40680b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40680afc60 0x7f40680b2170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f407c069e50 tx=0x7f4078005c10 comp rx=0 tx=0).stop 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 msgr2=0x7f407c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 0x7f407c19a750 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d740 tx=0x7f406c00dc00 comp rx=0 tx=0).stop 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 shutdown_connections 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f40680afc60 0x7f40680b2170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f407c10d3a0 0x7f407c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f407c108a40 0x7f407c19acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 --2- 172.21.15.112:0/4126900942 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f407c103280 0x7f407c19a750 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 >> 172.21.15.112:0/4126900942 conn(0x7f407c077880 msgr2=0x7f407c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:45.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 shutdown_connections 2024-04-25T16:54:45.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.307+0000 7f40847b2640 1 -- 172.21.15.112:0/4126900942 wait complete. 2024-04-25T16:54:45.316 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T16:54:45.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 -- 172.21.15.112:0/4083924651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc41810e050 msgr2=0x7fc418110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/4083924651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc41810e050 0x7fc418110540 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc408009ca0 tx=0x7fc40802a950 comp rx=0 tx=0).stop 2024-04-25T16:54:45.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 -- 172.21.15.112:0/4083924651 shutdown_connections 2024-04-25T16:54:45.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/4083924651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc41810e050 0x7fc418110540 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/4083924651 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 0x7fc41810db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/4083924651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc418104c90 0x7fc4181050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 -- 172.21.15.112:0/4083924651 >> 172.21.15.112:0/4083924651 conn(0x7fc418078100 msgr2=0x7fc418078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 -- 172.21.15.112:0/4083924651 shutdown_connections 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.433+0000 7fc41ce0c640 1 -- 172.21.15.112:0/4083924651 wait complete. 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 Processor -- start 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 -- start start 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc418104c90 0x7fc418102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 0x7fc418102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41810e050 0x7fc418102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc418113cf0 con 0x7fc418104c90 2024-04-25T16:54:45.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc418113b70 con 0x7fc41810b6f0 2024-04-25T16:54:45.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41ce0c640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc418113e70 con 0x7fc41810e050 2024-04-25T16:54:45.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc418104c90 0x7fc418102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc416d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41810e050 0x7fc418102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc415d9b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 0x7fc418102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc418104c90 0x7fc418102550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:40494/0 (socket says 172.21.15.112:40494) 2024-04-25T16:54:45.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc416d9d640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41810e050 0x7fc418102fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.112:37416/0 (socket says 172.21.15.112:37416) 2024-04-25T16:54:45.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.434+0000 7fc41659c640 1 -- 172.21.15.112:0/3686525780 learned_addr learned my addr 172.21.15.112:0/3686525780 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc415d9b640 1 -- 172.21.15.112:0/3686525780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41810e050 msgr2=0x7fc418102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc415d9b640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41810e050 0x7fc418102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc415d9b640 1 -- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc418104c90 msgr2=0x7fc418102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc415d9b640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc418104c90 0x7fc418102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc415d9b640 1 -- 172.21.15.112:0/3686525780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4181a8ad0 con 0x7fc41810b6f0 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc415d9b640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 0x7fc418102a90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc40000e9f0 tx=0x7fc40000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000ccc0 con 0x7fc41810b6f0 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4000108e0 con 0x7fc41810b6f0 2024-04-25T16:54:45.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4181a8dc0 con 0x7fc41810b6f0 2024-04-25T16:54:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400018880 con 0x7fc41810b6f0 2024-04-25T16:54:45.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.435+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181a9280 con 0x7fc41810b6f0 2024-04-25T16:54:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.436+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41806afa0 con 0x7fc41810b6f0 2024-04-25T16:54:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.438+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fc400010430 con 0x7fc41810b6f0 2024-04-25T16:54:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.439+0000 7fc40f7fe640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc4040afc60 0x7fc4040b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.439+0000 7fc41659c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc4040afc60 0x7fc4040b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.439+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7fc4000df7f0 con 0x7fc41810b6f0 2024-04-25T16:54:45.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.439+0000 7fc41659c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc4040afc60 0x7fc4040b2170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc410009a40 tx=0x7fc4100091c0 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:45.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.443+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4000a79f0 con 0x7fc41810b6f0 2024-04-25T16:54:45.448 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:45 smithi133 ceph-mon[32691]: pgmap v520: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 61 op/s 2024-04-25T16:54:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:45 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3098221952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:45.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:45 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4126900942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:54:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:45 smithi112 ceph-mon[29122]: pgmap v520: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 61 op/s 2024-04-25T16:54:45.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:45 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3098221952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:45.595 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:45 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4126900942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:54:45.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.623+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc4181a94b0 con 0x7fc41810b6f0 2024-04-25T16:54:45.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.624+0000 7fc40f7fe640 1 -- 172.21.15.112:0/3686525780 <== mon.2 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7fc4000abcf0 con 0x7fc41810b6f0 2024-04-25T16:54:45.622 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T16:54:45.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc4040afc60 msgr2=0x7fc4040b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc4040afc60 0x7fc4040b2170 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc410009a40 tx=0x7fc4100091c0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 msgr2=0x7fc418102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 0x7fc418102a90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc40000e9f0 tx=0x7fc40000eeb0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 shutdown_connections 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fc4040afc60 0x7fc4040b2170 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41810e050 0x7fc418102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc41810b6f0 0x7fc418102a90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 --2- 172.21.15.112:0/3686525780 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc418104c90 0x7fc418102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 >> 172.21.15.112:0/3686525780 conn(0x7fc418078100 msgr2=0x7fc418101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 shutdown_connections 2024-04-25T16:54:45.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.626+0000 7fc41ce0c640 1 -- 172.21.15.112:0/3686525780 wait complete. 2024-04-25T16:54:45.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T16:54:45.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 -- 172.21.15.112:0/2812937060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c103830 msgr2=0x7fda3c10f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 --2- 172.21.15.112:0/2812937060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c103830 0x7fda3c10f530 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fda34009ca0 tx=0x7fda3402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 -- 172.21.15.112:0/2812937060 shutdown_connections 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 --2- 172.21.15.112:0/2812937060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c103830 0x7fda3c10f530 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 --2- 172.21.15.112:0/2812937060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda3c108eb0 0x7fda3c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 --2- 172.21.15.112:0/2812937060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda3c108a10 0x7fda3c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 -- 172.21.15.112:0/2812937060 >> 172.21.15.112:0/2812937060 conn(0x7fda3c077880 msgr2=0x7fda3c077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:45.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 -- 172.21.15.112:0/2812937060 shutdown_connections 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.748+0000 7fda42b72640 1 -- 172.21.15.112:0/2812937060 wait complete. 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 Processor -- start 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 -- start start 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda3c103830 0x7fda3c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 0x7fda3c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda3c108eb0 0x7fda3c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c113d20 con 0x7fda3c108a10 2024-04-25T16:54:45.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c113ba0 con 0x7fda3c103830 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda42b72640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c113ea0 con 0x7fda3c108eb0 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda3bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 0x7fda3c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda3bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 0x7fda3c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56292/0 (socket says 172.21.15.112:56292) 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda3bfff640 1 -- 172.21.15.112:0/481016947 learned_addr learned my addr 172.21.15.112:0/481016947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.749+0000 7fda4110f640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda3c108eb0 0x7fda3c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda4090e640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda3c103830 0x7fda3c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda3bfff640 1 -- 172.21.15.112:0/481016947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda3c108eb0 msgr2=0x7fda3c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda3bfff640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda3c108eb0 0x7fda3c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda3bfff640 1 -- 172.21.15.112:0/481016947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda3c103830 msgr2=0x7fda3c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda3bfff640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda3c103830 0x7fda3c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda3bfff640 1 -- 172.21.15.112:0/481016947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c19f580 con 0x7fda3c108a10 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda3bfff640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 0x7fda3c19ac80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fda3000c970 tx=0x7fda3000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda30004220 con 0x7fda3c108a10 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c19f870 con 0x7fda3c108a10 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda300043c0 con 0x7fda3c108a10 2024-04-25T16:54:45.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3000f450 con 0x7fda3c108a10 2024-04-25T16:54:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.750+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a71f0 con 0x7fda3c108a10 2024-04-25T16:54:45.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.751+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda3c103ea0 con 0x7fda3c108a10 2024-04-25T16:54:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.752+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fda300079b0 con 0x7fda3c108a10 2024-04-25T16:54:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.753+0000 7fda39ffb640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda240afc60 0x7fda240b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.753+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7fda300df180 con 0x7fda3c108a10 2024-04-25T16:54:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.753+0000 7fda4090e640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda240afc60 0x7fda240b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:45.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.753+0000 7fda4090e640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda240afc60 0x7fda240b2170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fda2c005fd0 tx=0x7fda2c005850 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:45.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.758+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fda30014030 con 0x7fda3c108a10 2024-04-25T16:54:45.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:45 smithi116 ceph-mon[32712]: pgmap v520: 97 pgs: 97 active+clean; 299 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 1.0 MiB/s wr, 61 op/s 2024-04-25T16:54:45.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:45 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3098221952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:45.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:45 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4126900942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:54:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.934+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fda3c10ab20 con 0x7fda3c108a10 2024-04-25T16:54:45.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.935+0000 7fda39ffb640 1 -- 172.21.15.112:0/481016947 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7fda300a79d0 con 0x7fda3c108a10 2024-04-25T16:54:45.933 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T16:54:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda240afc60 msgr2=0x7fda240b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda240afc60 0x7fda240b2170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fda2c005fd0 tx=0x7fda2c005850 comp rx=0 tx=0).stop 2024-04-25T16:54:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 msgr2=0x7fda3c19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 0x7fda3c19ac80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fda3000c970 tx=0x7fda3000ce30 comp rx=0 tx=0).stop 2024-04-25T16:54:45.935 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 shutdown_connections 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fda240afc60 0x7fda240b2170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda3c108eb0 0x7fda3c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda3c108a10 0x7fda3c19ac80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 --2- 172.21.15.112:0/481016947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda3c103830 0x7fda3c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 >> 172.21.15.112:0/481016947 conn(0x7fda3c077880 msgr2=0x7fda3c0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 shutdown_connections 2024-04-25T16:54:45.936 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:45.937+0000 7fda42b72640 1 -- 172.21.15.112:0/481016947 wait complete. 2024-04-25T16:54:45.946 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:54:46.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 -- 172.21.15.112:0/1113503098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108a30 msgr2=0x7f07901025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:46.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 --2- 172.21.15.112:0/1113503098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108a30 0x7f07901025e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0780009900 tx=0x7f078002a410 comp rx=0 tx=0).stop 2024-04-25T16:54:46.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 -- 172.21.15.112:0/1113503098 shutdown_connections 2024-04-25T16:54:46.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 --2- 172.21.15.112:0/1113503098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0790103190 0x7f079010f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 --2- 172.21.15.112:0/1113503098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0790108ed0 0x7f0790102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 --2- 172.21.15.112:0/1113503098 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108a30 0x7f07901025e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 -- 172.21.15.112:0/1113503098 >> 172.21.15.112:0/1113503098 conn(0x7f07900fe260 msgr2=0x7f0790100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 -- 172.21.15.112:0/1113503098 shutdown_connections 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 -- 172.21.15.112:0/1113503098 wait complete. 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.058+0000 7f07961b2640 1 Processor -- start 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 -- start start 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0790103190 0x7f0790199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0790108a30 0x7f0790199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 0x7f079019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0790112c30 con 0x7f0790108ed0 2024-04-25T16:54:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0790112ab0 con 0x7f0790108a30 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f07961b2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0790112db0 con 0x7f0790103190 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 0x7f079019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 0x7f079019de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56314/0 (socket says 172.21.15.112:56314) 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 -- 172.21.15.112:0/3330710958 learned_addr learned my addr 172.21.15.112:0/3330710958 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078effd640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0790108a30 0x7f0790199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078f7fe640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0790103190 0x7f0790199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 -- 172.21.15.112:0/3330710958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0790103190 msgr2=0x7f0790199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0790103190 0x7f0790199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 -- 172.21.15.112:0/3330710958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0790108a30 msgr2=0x7f0790199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0790108a30 0x7f0790199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 -- 172.21.15.112:0/3330710958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079019e490 con 0x7f0790108ed0 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.059+0000 7f078ffff640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 0x7f079019de50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f078800eaa0 tx=0x7f078800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.060+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078800cdb0 con 0x7f0790108ed0 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.060+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0788005020 con 0x7f0790108ed0 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.060+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07880116d0 con 0x7f0790108ed0 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.060+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079019e780 con 0x7f0790108ed0 2024-04-25T16:54:46.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.060+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07901a6100 con 0x7f0790108ed0 2024-04-25T16:54:46.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.061+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079006a160 con 0x7f0790108ed0 2024-04-25T16:54:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.062+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f07880051d0 con 0x7f0790108ed0 2024-04-25T16:54:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.062+0000 7f078cff9640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f07780afc60 0x7f07780b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.063+0000 7f078f7fe640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f07780afc60 0x7f07780b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.063+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7f07880df5f0 con 0x7f0790108ed0 2024-04-25T16:54:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.063+0000 7f078f7fe640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f07780afc60 0x7f07780b2170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0780009230 tx=0x7f0780031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:46.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.066+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0788016030 con 0x7f0790108ed0 2024-04-25T16:54:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.255+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f079019a980 con 0x7f0790108ed0 2024-04-25T16:54:46.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.255+0000 7f078cff9640 1 -- 172.21.15.112:0/3330710958 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v109) ==== 81+0+956 (secure 0 0 0) 0x7f079019a980 con 0x7f0790108ed0 2024-04-25T16:54:46.253 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/3947590362 2024-04-26T16:38:27.501574+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/3947590362 2024-04-26T16:38:27.501574+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/386718339 2024-04-26T16:38:27.501574+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/4216056022 2024-04-26T16:38:27.501574+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/795264311 2024-04-26T16:38:27.501574+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3651646348 2024-04-26T16:28:48.092289+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/108124461 2024-04-26T16:29:14.986280+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1519283708 2024-04-26T16:28:48.092289+0000 2024-04-25T16:54:46.254 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/2445069646 2024-04-26T16:38:27.501574+0000 2024-04-25T16:54:46.255 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/944410496 2024-04-26T16:28:48.092289+0000 2024-04-25T16:54:46.255 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/1130097819 2024-04-26T16:28:48.092289+0000 2024-04-25T16:54:46.255 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/3285899721 2024-04-26T16:29:14.986280+0000 2024-04-25T16:54:46.255 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6800/108124461 2024-04-26T16:29:14.986280+0000 2024-04-25T16:54:46.257 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:6801/1130097819 2024-04-26T16:28:48.092289+0000 2024-04-25T16:54:46.257 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/404948490 2024-04-26T16:29:14.986280+0000 2024-04-25T16:54:46.257 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.15.112:0/1942519556 2024-04-26T16:29:14.986280+0000 2024-04-25T16:54:46.258 INFO:teuthology.orchestra.run.smithi112.stderr:listed 16 entries 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f07780afc60 msgr2=0x7f07780b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f07780afc60 0x7f07780b2170 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0780009230 tx=0x7f0780031400 comp rx=0 tx=0).stop 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 msgr2=0x7f079019de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 0x7f079019de50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f078800eaa0 tx=0x7f078800ef60 comp rx=0 tx=0).stop 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 shutdown_connections 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f07780afc60 0x7f07780b2170 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0790108ed0 0x7f079019de50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0790108a30 0x7f0790199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 --2- 172.21.15.112:0/3330710958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0790103190 0x7f0790199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.257+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 >> 172.21.15.112:0/3330710958 conn(0x7f07900fe260 msgr2=0x7f07900fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.258+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 shutdown_connections 2024-04-25T16:54:46.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:46.258+0000 7f07961b2640 1 -- 172.21.15.112:0/3330710958 wait complete. 2024-04-25T16:54:46.266 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T16:54:46.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:54:46.267 DEBUG:teuthology.orchestra.run.smithi112:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T16:54:46.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:46 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3686525780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:54:46.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:46 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/481016947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:54:46.594 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:46 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3330710958' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:54:46.687 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T16:54:46.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:54:46.688 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T16:54:46.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:46 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3686525780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:54:46.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:46 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/481016947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:54:46.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:46 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3330710958' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:54:46.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:54:46.738 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T16:54:46.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:46 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3686525780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T16:54:46.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:46 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/481016947' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T16:54:46.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:46 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3330710958' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:54:46.774 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T16:54:46.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:54:46.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T16:54:46.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:54:47.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T16:54:47.435 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T16:54:47.447 INFO:tasks.cephadm:Teardown begin 2024-04-25T16:54:47.447 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T16:54:47.448 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:54:47.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 -- 172.21.15.112:0/1885046901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 msgr2=0x7f4258102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 --2- 172.21.15.112:0/1885046901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258102f00 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4248009900 tx=0x7f424802a410 comp rx=0 tx=0).stop 2024-04-25T16:54:47.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 -- 172.21.15.112:0/1885046901 shutdown_connections 2024-04-25T16:54:47.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 --2- 172.21.15.112:0/1885046901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f425810d380 0x7f425810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 --2- 172.21.15.112:0/1885046901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258102f00 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 --2- 172.21.15.112:0/1885046901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4258108a20 0x7f42581025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 -- 172.21.15.112:0/1885046901 >> 172.21.15.112:0/1885046901 conn(0x7f4258078110 msgr2=0x7f4258078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 -- 172.21.15.112:0/1885046901 shutdown_connections 2024-04-25T16:54:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.562+0000 7f425d49f640 1 -- 172.21.15.112:0/1885046901 wait complete. 2024-04-25T16:54:47.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 Processor -- start 2024-04-25T16:54:47.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 -- start start 2024-04-25T16:54:47.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4258108a20 0x7f4258199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f425810d380 0x7f425819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4258112c30 con 0x7f4258102af0 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4258112ab0 con 0x7f4258108a20 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f425d49f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4258112db0 con 0x7f425810d380 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56326/0 (socket says 172.21.15.112:56326) 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 -- 172.21.15.112:0/3637014061 learned_addr learned my addr 172.21.15.112:0/3637014061 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:47.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f42567fc640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4258108a20 0x7f4258199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f42577fe640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f425810d380 0x7f425819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 -- 172.21.15.112:0/3637014061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f425810d380 msgr2=0x7f425819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f425810d380 0x7f425819de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 -- 172.21.15.112:0/3637014061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4258108a20 msgr2=0x7f4258199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4258108a20 0x7f4258199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.563+0000 7f4256ffd640 1 -- 172.21.15.112:0/3637014061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425819e470 con 0x7f4258102af0 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.564+0000 7f4256ffd640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258199610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f424000d740 tx=0x7f424000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:47.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.564+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240017070 con 0x7f4258102af0 2024-04-25T16:54:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.564+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f424000de50 con 0x7f4258102af0 2024-04-25T16:54:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.564+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425819e760 con 0x7f4258102af0 2024-04-25T16:54:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.564+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240004ce0 con 0x7f4258102af0 2024-04-25T16:54:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.564+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425819ec30 con 0x7f4258102af0 2024-04-25T16:54:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.565+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4258103cd0 con 0x7f4258102af0 2024-04-25T16:54:47.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.566+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f4240004e80 con 0x7f4258102af0 2024-04-25T16:54:47.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.567+0000 7f424ffff640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f42440afc60 0x7f42440b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.567+0000 7f42567fc640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f42440afc60 0x7f42440b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.567+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7f42400ded70 con 0x7f4258102af0 2024-04-25T16:54:47.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.567+0000 7f42567fc640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f42440afc60 0x7f42440b2170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f424802aa00 tx=0x7f4248005a20 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:47.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.571+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f42400a6f70 con 0x7f4258102af0 2024-04-25T16:54:47.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:47 smithi133 ceph-mon[32691]: pgmap v521: 97 pgs: 97 active+clean; 300 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 726 KiB/s wr, 47 op/s 2024-04-25T16:54:47.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:47 smithi116 ceph-mon[32712]: pgmap v521: 97 pgs: 97 active+clean; 300 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 726 KiB/s wr, 47 op/s 2024-04-25T16:54:47.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.786+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f425806afa0 con 0x7f4258102af0 2024-04-25T16:54:47.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.787+0000 7f424ffff640 1 -- 172.21.15.112:0/3637014061 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11880 (secure 0 0 0) 0x7f42400ab270 con 0x7f4258102af0 2024-04-25T16:54:47.784 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:54:47.785 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":16071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1317678143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1317678143},{"type":"v1","addr":"172.21.15.116:6835","nonce":1317678143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":16509,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1994270531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1994270531},{"type":"v1","addr":"172.21.15.116:6833","nonce":1994270531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:54:30.398548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24560,24514,24524]},"id":1}]} 2024-04-25T16:54:47.785 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f42440afc60 msgr2=0x7f42440b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f42440afc60 0x7f42440b2170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f424802aa00 tx=0x7f4248005a20 comp rx=0 tx=0).stop 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 msgr2=0x7f4258199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258199610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f424000d740 tx=0x7f424000dc00 comp rx=0 tx=0).stop 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 shutdown_connections 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f42440afc60 0x7f42440b2170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f425810d380 0x7f425819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4258108a20 0x7f4258199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 --2- 172.21.15.112:0/3637014061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4258102af0 0x7f4258199610 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 >> 172.21.15.112:0/3637014061 conn(0x7f4258078110 msgr2=0x7f42581040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 shutdown_connections 2024-04-25T16:54:47.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.789+0000 7f425d49f640 1 -- 172.21.15.112:0/3637014061 wait complete. 2024-04-25T16:54:47.796 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:47 smithi112 ceph-mon[29122]: pgmap v521: 97 pgs: 97 active+clean; 300 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 726 KiB/s wr, 47 op/s 2024-04-25T16:54:47.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:54:47.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 -- 172.21.15.112:0/603251996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10e040 msgr2=0x7f50fc110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 --2- 172.21.15.112:0/603251996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10e040 0x7f50fc110530 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009900 tx=0x7f50f402a410 comp rx=0 tx=0).stop 2024-04-25T16:54:47.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 -- 172.21.15.112:0/603251996 shutdown_connections 2024-04-25T16:54:47.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 --2- 172.21.15.112:0/603251996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10b650 0x7f50fc10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 --2- 172.21.15.112:0/603251996 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10e040 0x7f50fc110530 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 --2- 172.21.15.112:0/603251996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 0x7f50fc105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 -- 172.21.15.112:0/603251996 >> 172.21.15.112:0/603251996 conn(0x7f50fc078110 msgr2=0x7f50fc078550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 -- 172.21.15.112:0/603251996 shutdown_connections 2024-04-25T16:54:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 -- 172.21.15.112:0/603251996 wait complete. 2024-04-25T16:54:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.909+0000 7f510169a640 1 Processor -- start 2024-04-25T16:54:47.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 -- start start 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 0x7f50fc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 0x7f50fc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10e040 0x7f50fc19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113ca0 con 0x7f50fc10b650 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113b20 con 0x7f50fc104bf0 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f510169a640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113e20 con 0x7f50fc10e040 2024-04-25T16:54:47.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 0x7f50fc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 0x7f50fc19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56356/0 (socket says 172.21.15.112:56356) 2024-04-25T16:54:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 -- 172.21.15.112:0/2772758417 learned_addr learned my addr 172.21.15.112:0/2772758417 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50faffd640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 0x7f50fc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 -- 172.21.15.112:0/2772758417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10e040 msgr2=0x7f50fc19eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fb7fe640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10e040 0x7f50fc19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10e040 0x7f50fc19eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 -- 172.21.15.112:0/2772758417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 msgr2=0x7f50fc19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:47.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 0x7f50fc19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:47.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fa7fc640 1 -- 172.21.15.112:0/2772758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc19f460 con 0x7f50fc10b650 2024-04-25T16:54:47.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50faffd640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 0x7f50fc19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:54:47.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.910+0000 7f50fb7fe640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10e040 0x7f50fc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:54:47.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.911+0000 7f50fa7fc640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 0x7f50fc19abe0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f50e400d7a0 tx=0x7f50e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.911+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4017070 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.911+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc19f750 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.911+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e40054d0 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.911+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4004c10 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.911+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc19fc50 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.912+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc06a770 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.913+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f50e4004db0 con 0x7f50fc10b650 2024-04-25T16:54:47.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.914+0000 7f50f3fff640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f50e80afc60 0x7f50e80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:47.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.914+0000 7f50faffd640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f50e80afc60 0x7f50e80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:47.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.914+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7f50e40deed0 con 0x7f50fc10b650 2024-04-25T16:54:47.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.914+0000 7f50faffd640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f50e80afc60 0x7f50e80b2170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50ec007e70 tx=0x7f50ec007850 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:47.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:47.918+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50e40a7720 con 0x7f50fc10b650 2024-04-25T16:54:48.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.133+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50fc1a71c0 con 0x7f50fc10b650 2024-04-25T16:54:48.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.134+0000 7f50f3fff640 1 -- 172.21.15.112:0/2772758417 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11880 (secure 0 0 0) 0x7f50e40a7020 con 0x7f50fc10b650 2024-04-25T16:54:48.131 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:54:48.131 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14547,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/759452882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":759452882},{"type":"v1","addr":"172.21.15.112:6835","nonce":759452882}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1910235790","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6836","nonce":1910235790},{"type":"v1","addr":"172.21.15.112:6837","nonce":1910235790}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14628,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6839/72433575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6838","nonce":72433575},{"type":"v1","addr":"172.21.15.112:6839","nonce":72433575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6841/4208559487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6840","nonce":4208559487},{"type":"v1","addr":"172.21.15.112:6841","nonce":4208559487}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":16071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1317678143","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1317678143},{"type":"v1","addr":"172.21.15.116:6835","nonce":1317678143}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":16509,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1994270531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1994270531},{"type":"v1","addr":"172.21.15.116:6833","nonce":1994270531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T16:36:26.563199+0000","modified":"2024-04-25T16:54:30.398548+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24524,"mds_2":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24421":{"gid":24421,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6833/2075237111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":2075237111},{"type":"v1","addr":"172.21.15.133:6833","nonce":2075237111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6835/2304762610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2304762610},{"type":"v1","addr":"172.21.15.133:6835","nonce":2304762610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24484":{"gid":24484,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.133:6837/3840802165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":3840802165},{"type":"v1","addr":"172.21.15.133:6837","nonce":3840802165}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6839/99930432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":99930432},{"type":"v1","addr":"172.21.15.133:6839","nonce":99930432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"h","rank":1,"incarnation":17,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6837/797562383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6836","nonce":797562383},{"type":"v1","addr":"172.21.15.116:6837","nonce":797562383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6839/1016899540","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6838","nonce":1016899540},{"type":"v1","addr":"172.21.15.116:6839","nonce":1016899540}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24560,"qdb_cluster":[24560,24514,24524]},"id":1}]} 2024-04-25T16:54:48.132 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 40 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f50e80afc60 msgr2=0x7f50e80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f50e80afc60 0x7f50e80b2170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50ec007e70 tx=0x7f50ec007850 comp rx=0 tx=0).stop 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 msgr2=0x7f50fc19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 0x7f50fc19abe0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f50e400d7a0 tx=0x7f50e400dc60 comp rx=0 tx=0).stop 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 shutdown_connections 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f50e80afc60 0x7f50e80b2170 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50fc10e040 0x7f50fc19eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50fc10b650 0x7f50fc19abe0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 --2- 172.21.15.112:0/2772758417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f50fc104bf0 0x7f50fc19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 >> 172.21.15.112:0/2772758417 conn(0x7f50fc078110 msgr2=0x7f50fc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 shutdown_connections 2024-04-25T16:54:48.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.136+0000 7f510169a640 1 -- 172.21.15.112:0/2772758417 wait complete. 2024-04-25T16:54:48.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:54:48.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3771458779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 msgr2=0x7f4cd8102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3771458779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8102f70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009900 tx=0x7f4cd402a410 comp rx=0 tx=0).stop 2024-04-25T16:54:48.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3771458779 shutdown_connections 2024-04-25T16:54:48.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3771458779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cd810d3a0 0x7f4cd810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3771458779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8102f70 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3771458779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd8108a40 0x7f4cd8102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3771458779 >> 172.21.15.112:0/3771458779 conn(0x7f4cd80fe260 msgr2=0x7f4cd8100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3771458779 shutdown_connections 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.256+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3771458779 wait complete. 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 Processor -- start 2024-04-25T16:54:48.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 -- start start 2024-04-25T16:54:48.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd8108a40 0x7f4cd8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cd810d3a0 0x7f4cd819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8112c00 con 0x7f4cd8102b60 2024-04-25T16:54:48.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8112a80 con 0x7f4cd810d3a0 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdfcf2640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8112d80 con 0x7f4cd8108a40 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56380/0 (socket says 172.21.15.112:56380) 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 -- 172.21.15.112:0/3258810276 learned_addr learned my addr 172.21.15.112:0/3258810276 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdd28d640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd8108a40 0x7f4cd8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cde28f640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cd810d3a0 0x7f4cd819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 -- 172.21.15.112:0/3258810276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd8108a40 msgr2=0x7f4cd8199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd8108a40 0x7f4cd8199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 -- 172.21.15.112:0/3258810276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cd810d3a0 msgr2=0x7f4cd819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.257+0000 7f4cdda8e640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cd810d3a0 0x7f4cd819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cdda8e640 1 -- 172.21.15.112:0/3258810276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd819e460 con 0x7f4cd8102b60 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cdda8e640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8199600 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800d740 tx=0x7f4cc800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8017070 con 0x7f4cd8102b60 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cc800de50 con 0x7f4cd8102b60 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd819e750 con 0x7f4cd8102b60 2024-04-25T16:54:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8004ce0 con 0x7f4cd8102b60 2024-04-25T16:54:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.258+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd819ec20 con 0x7f4cd8102b60 2024-04-25T16:54:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.259+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd806a190 con 0x7f4cd8102b60 2024-04-25T16:54:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.261+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f4cc8004e80 con 0x7f4cd8102b60 2024-04-25T16:54:48.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.262+0000 7f4cceffd640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4cc40afc60 0x7f4cc40b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.262+0000 7f4cdd28d640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4cc40afc60 0x7f4cc40b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.262+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7f4cc80ded70 con 0x7f4cd8102b60 2024-04-25T16:54:48.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.262+0000 7f4cdd28d640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4cc40afc60 0x7f4cc40b2170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009230 tx=0x7f4cd4031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:48.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.265+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4cc80a6f70 con 0x7f4cd8102b60 2024-04-25T16:54:48.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.452+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cd8106f30 con 0x7f4cd8102b60 2024-04-25T16:54:48.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.454+0000 7f4cceffd640 1 -- 172.21.15.112:0/3258810276 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+23347 (secure 0 0 0) 0x7f4cc80ab270 con 0x7f4cd8102b60 2024-04-25T16:54:48.452 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T16:54:48.452 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":109,"fsid":"a7905982-0320-11ef-bc93-c7b262605968","created":"2024-04-25T16:28:12.063898+0000","modified":"2024-04-25T16:54:17.171776+0000","last_up_change":"2024-04-25T16:35:35.139851+0000","last_in_change":"2024-04-25T16:35:15.215644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T16:31:41.074159+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T16:36:24.204313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T16:36:25.525471+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2011d44b-d406-413a-aa94-1a3a50fcde37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6803","nonce":2222102236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6805","nonce":2222102236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6809","nonce":2222102236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":2222102236},{"type":"v1","addr":"172.21.15.112:6807","nonce":2222102236}]},"public_addr":"172.21.15.112:6803/2222102236","cluster_addr":"172.21.15.112:6805/2222102236","heartbeat_back_addr":"172.21.15.112:6809/2222102236","heartbeat_front_addr":"172.21.15.112:6807/2222102236","state":["exists","up"]},{"osd":1,"uuid":"cb335e86-8675-4ab0-826c-9cf84a05a605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6801","nonce":2666280069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6803","nonce":2666280069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6807","nonce":2666280069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":2666280069},{"type":"v1","addr":"172.21.15.116:6805","nonce":2666280069}]},"public_addr":"172.21.15.116:6801/2666280069","cluster_addr":"172.21.15.116:6803/2666280069","heartbeat_back_addr":"172.21.15.116:6807/2666280069","heartbeat_front_addr":"172.21.15.116:6805/2666280069","state":["exists","up"]},{"osd":2,"uuid":"a075e9d7-1972-4815-8471-6a49816ec39f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6801","nonce":1571915976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6803","nonce":1571915976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6807","nonce":1571915976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1571915976},{"type":"v1","addr":"172.21.15.133:6805","nonce":1571915976}]},"public_addr":"172.21.15.133:6801/1571915976","cluster_addr":"172.21.15.133:6803/1571915976","heartbeat_back_addr":"172.21.15.133:6807/1571915976","heartbeat_front_addr":"172.21.15.133:6805/1571915976","state":["exists","up"]},{"osd":3,"uuid":"1d45175e-d218-4c74-bcec-a037fcb395f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6811","nonce":1393578642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6813","nonce":1393578642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6817","nonce":1393578642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1393578642},{"type":"v1","addr":"172.21.15.112:6815","nonce":1393578642}]},"public_addr":"172.21.15.112:6811/1393578642","cluster_addr":"172.21.15.112:6813/1393578642","heartbeat_back_addr":"172.21.15.112:6817/1393578642","heartbeat_front_addr":"172.21.15.112:6815/1393578642","state":["exists","up"]},{"osd":4,"uuid":"a0f9a634-8f5a-4dbe-a8e9-8f74872cb417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6809","nonce":2551746095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6811","nonce":2551746095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6815","nonce":2551746095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":2551746095},{"type":"v1","addr":"172.21.15.116:6813","nonce":2551746095}]},"public_addr":"172.21.15.116:6809/2551746095","cluster_addr":"172.21.15.116:6811/2551746095","heartbeat_back_addr":"172.21.15.116:6815/2551746095","heartbeat_front_addr":"172.21.15.116:6813/2551746095","state":["exists","up"]},{"osd":5,"uuid":"3a17fcaf-3f2f-4fef-b666-a6c9927dc38a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6809","nonce":3110277631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6811","nonce":3110277631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6815","nonce":3110277631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3110277631},{"type":"v1","addr":"172.21.15.133:6813","nonce":3110277631}]},"public_addr":"172.21.15.133:6809/3110277631","cluster_addr":"172.21.15.133:6811/3110277631","heartbeat_back_addr":"172.21.15.133:6815/3110277631","heartbeat_front_addr":"172.21.15.133:6813/3110277631","state":["exists","up"]},{"osd":6,"uuid":"bd1c866b-efbc-45fe-9c88-d9e0e2f8fb29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6819","nonce":4240993708}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6821","nonce":4240993708}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6825","nonce":4240993708}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":4240993708},{"type":"v1","addr":"172.21.15.112:6823","nonce":4240993708}]},"public_addr":"172.21.15.112:6819/4240993708","cluster_addr":"172.21.15.112:6821/4240993708","heartbeat_back_addr":"172.21.15.112:6825/4240993708","heartbeat_front_addr":"172.21.15.112:6823/4240993708","state":["exists","up"]},{"osd":7,"uuid":"79c1b323-2808-4b81-9164-36836a690a83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6817","nonce":590137148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6819","nonce":590137148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6823","nonce":590137148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":590137148},{"type":"v1","addr":"172.21.15.116:6821","nonce":590137148}]},"public_addr":"172.21.15.116:6817/590137148","cluster_addr":"172.21.15.116:6819/590137148","heartbeat_back_addr":"172.21.15.116:6823/590137148","heartbeat_front_addr":"172.21.15.116:6821/590137148","state":["exists","up"]},{"osd":8,"uuid":"c5b8b39f-b8a4-4ee1-800f-0ab3e68f3e0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6817","nonce":1430124311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6819","nonce":1430124311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6823","nonce":1430124311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":1430124311},{"type":"v1","addr":"172.21.15.133:6821","nonce":1430124311}]},"public_addr":"172.21.15.133:6817/1430124311","cluster_addr":"172.21.15.133:6819/1430124311","heartbeat_back_addr":"172.21.15.133:6823/1430124311","heartbeat_front_addr":"172.21.15.133:6821/1430124311","state":["exists","up"]},{"osd":9,"uuid":"a29c6af5-9461-4eef-b445-bcb8a7bfed1e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6827","nonce":2630590642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6829","nonce":2630590642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6833","nonce":2630590642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":2630590642},{"type":"v1","addr":"172.21.15.112:6831","nonce":2630590642}]},"public_addr":"172.21.15.112:6827/2630590642","cluster_addr":"172.21.15.112:6829/2630590642","heartbeat_back_addr":"172.21.15.112:6833/2630590642","heartbeat_front_addr":"172.21.15.112:6831/2630590642","state":["exists","up"]},{"osd":10,"uuid":"b0d3168a-8529-49d8-bd4c-1c1bd7336f60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6825","nonce":3599339690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6827","nonce":3599339690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6831","nonce":3599339690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":3599339690},{"type":"v1","addr":"172.21.15.116:6829","nonce":3599339690}]},"public_addr":"172.21.15.116:6825/3599339690","cluster_addr":"172.21.15.116:6827/3599339690","heartbeat_back_addr":"172.21.15.116:6831/3599339690","heartbeat_front_addr":"172.21.15.116:6829/3599339690","state":["exists","up"]},{"osd":11,"uuid":"f41bd871-e7f8-4a6b-b3a4-e77aa1feaed1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6825","nonce":2381034021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6827","nonce":2381034021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6831","nonce":2381034021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":2381034021},{"type":"v1","addr":"172.21.15.133:6829","nonce":2381034021}]},"public_addr":"172.21.15.133:6825/2381034021","cluster_addr":"172.21.15.133:6827/2381034021","heartbeat_back_addr":"172.21.15.133:6831/2381034021","heartbeat_front_addr":"172.21.15.133:6829/2381034021","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:30:50.638351+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:14.892075+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:31:38.537408+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:03.949951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:29.434060+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:32:54.836679+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:21.268522+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:33:46.951055+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:12.994527+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:34:40.139251+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:06.742760+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T16:35:33.982067+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.1b","mappings":[{"from":3,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6801/3947590362":"2024-04-26T16:38:27.501574+0000","172.21.15.112:6800/3947590362":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/386718339":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/4216056022":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/795264311":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/3651646348":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6801/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1519283708":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/2445069646":"2024-04-26T16:38:27.501574+0000","172.21.15.112:0/944410496":"2024-04-26T16:28:48.092289+0000","172.21.15.112:6800/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/3285899721":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6800/108124461":"2024-04-26T16:29:14.986280+0000","172.21.15.112:6801/1130097819":"2024-04-26T16:28:48.092289+0000","172.21.15.112:0/404948490":"2024-04-26T16:29:14.986280+0000","172.21.15.112:0/1942519556":"2024-04-26T16:29:14.986280+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:54:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4cc40afc60 msgr2=0x7f4cc40b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4cc40afc60 0x7f4cc40b2170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009230 tx=0x7f4cd4031400 comp rx=0 tx=0).stop 2024-04-25T16:54:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 msgr2=0x7f4cd8199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8199600 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800d740 tx=0x7f4cc800dc00 comp rx=0 tx=0).stop 2024-04-25T16:54:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 shutdown_connections 2024-04-25T16:54:48.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f4cc40afc60 0x7f4cc40b2170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cd8108a40 0x7f4cd8199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4cd810d3a0 0x7f4cd819de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 --2- 172.21.15.112:0/3258810276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd8102b60 0x7f4cd8199600 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 >> 172.21.15.112:0/3258810276 conn(0x7f4cd80fe260 msgr2=0x7f4cd81040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 shutdown_connections 2024-04-25T16:54:48.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.456+0000 7f4cdfcf2640 1 -- 172.21.15.112:0/3258810276 wait complete. 2024-04-25T16:54:48.465 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T16:54:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 -- 172.21.15.112:0/3389241737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108e60 msgr2=0x7fcfa0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 --2- 172.21.15.112:0/3389241737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108e60 0x7fcfa0103280 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009900 tx=0x7fcf9402a480 comp rx=0 tx=0).stop 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 -- 172.21.15.112:0/3389241737 shutdown_connections 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 --2- 172.21.15.112:0/3389241737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfa0103910 0x7fcfa010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 --2- 172.21.15.112:0/3389241737 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108e60 0x7fcfa0103280 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 --2- 172.21.15.112:0/3389241737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa0108a50 0x7fcfa0102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 -- 172.21.15.112:0/3389241737 >> 172.21.15.112:0/3389241737 conn(0x7fcfa0077880 msgr2=0x7fcfa0077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 -- 172.21.15.112:0/3389241737 shutdown_connections 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.582+0000 7fcfa638f640 1 -- 172.21.15.112:0/3389241737 wait complete. 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 Processor -- start 2024-04-25T16:54:48.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 -- start start 2024-04-25T16:54:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 0x7fcfa019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa0108e60 0x7fcfa019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfa019ebb0 0x7fcfa019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0112c20 con 0x7fcfa0108a50 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0112aa0 con 0x7fcfa0108e60 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcfa638f640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa0112da0 con 0x7fcfa019ebb0 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 0x7fcfa019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.583+0000 7fcf9ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 0x7fcfa019a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56388/0 (socket says 172.21.15.112:56388) 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 -- 172.21.15.112:0/553605277 learned_addr learned my addr 172.21.15.112:0/553605277 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9f7fe640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa0108e60 0x7fcfa019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcfa492c640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfa019ebb0 0x7fcfa019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 -- 172.21.15.112:0/553605277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfa019ebb0 msgr2=0x7fcfa019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfa019ebb0 0x7fcfa019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 -- 172.21.15.112:0/553605277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa0108e60 msgr2=0x7fcfa019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa0108e60 0x7fcfa019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 -- 172.21.15.112:0/553605277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa019f600 con 0x7fcfa0108a50 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9ffff640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 0x7fcfa019a390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000d770 tx=0x7fcf9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90017070 con 0x7fcfa0108a50 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf90009c30 con 0x7fcfa0108a50 2024-04-25T16:54:48.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa019f8f0 con 0x7fcfa0108a50 2024-04-25T16:54:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9000f620 con 0x7fcfa0108a50 2024-04-25T16:54:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.584+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa01a7100 con 0x7fcfa0108a50 2024-04-25T16:54:48.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.586+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa00692e0 con 0x7fcfa0108a50 2024-04-25T16:54:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.587+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fcf9001a070 con 0x7fcfa0108a50 2024-04-25T16:54:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.588+0000 7fcf9d7fa640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcf880afce0 0x7fcf880b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.588+0000 7fcf9f7fe640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcf880afce0 0x7fcf880b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.588+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8600+0+0 (secure 0 0 0) 0x7fcf900de9c0 con 0x7fcfa0108a50 2024-04-25T16:54:48.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.588+0000 7fcf9f7fe640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcf880afce0 0x7fcf880b21f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf94000c00 tx=0x7fcf94031400 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:48.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.592+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf90009de0 con 0x7fcfa0108a50 2024-04-25T16:54:48.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3637014061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:54:48.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2772758417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:54:48.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3637014061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:54:48.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2772758417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:54:48.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.811+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcfa01a73e0 con 0x7fcfa0108a50 2024-04-25T16:54:48.810 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:48 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3637014061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:54:48.811 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:48 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2772758417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:54:48.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.814+0000 7fcf9d7fa640 1 -- 172.21.15.112:0/553605277 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) ==== 161+0+0 (secure 0 0 0) 0x7fcf900a6c40 con 0x7fcfa0108a50 2024-04-25T16:54:48.811 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcf880afce0 msgr2=0x7fcf880b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcf880afce0 0x7fcf880b21f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf94000c00 tx=0x7fcf94031400 comp rx=0 tx=0).stop 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 msgr2=0x7fcfa019a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 0x7fcfa019a390 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcf9000d770 tx=0x7fcf9000dc30 comp rx=0 tx=0).stop 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 shutdown_connections 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fcf880afce0 0x7fcf880b21f0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcf94000c00 tx=0x7fcf94031400 comp rx=0 tx=0).stop 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfa019ebb0 0x7fcfa019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcfa0108e60 0x7fcfa019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 --2- 172.21.15.112:0/553605277 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcfa0108a50 0x7fcfa019a390 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.817+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 >> 172.21.15.112:0/553605277 conn(0x7fcfa0077880 msgr2=0x7fcfa010e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.818+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 shutdown_connections 2024-04-25T16:54:48.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.818+0000 7fcfa638f640 1 -- 172.21.15.112:0/553605277 wait complete. 2024-04-25T16:54:48.827 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T16:54:48.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 -- 172.21.15.112:0/4085046798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418102af0 msgr2=0x7f1418102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 --2- 172.21.15.112:0/4085046798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418102af0 0x7f1418102f00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1408009900 tx=0x7f140802a410 comp rx=0 tx=0).stop 2024-04-25T16:54:48.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 -- 172.21.15.112:0/4085046798 shutdown_connections 2024-04-25T16:54:48.940 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 --2- 172.21.15.112:0/4085046798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f141810d380 0x7f141810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 --2- 172.21.15.112:0/4085046798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1418102af0 0x7f1418102f00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 --2- 172.21.15.112:0/4085046798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1418108a20 0x7f14181025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 -- 172.21.15.112:0/4085046798 >> 172.21.15.112:0/4085046798 conn(0x7f1418078100 msgr2=0x7f1418078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 -- 172.21.15.112:0/4085046798 shutdown_connections 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 -- 172.21.15.112:0/4085046798 wait complete. 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.942+0000 7f141df72640 1 Processor -- start 2024-04-25T16:54:48.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 -- start start 2024-04-25T16:54:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1418102af0 0x7f141819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1418108a20 0x7f141819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 0x7f141819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112be0 con 0x7f141810d380 2024-04-25T16:54:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112a60 con 0x7f1418102af0 2024-04-25T16:54:48.944 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f141df72640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112d60 con 0x7f1418108a20 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 0x7f141819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 0x7f141819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56396/0 (socket says 172.21.15.112:56396) 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 -- 172.21.15.112:0/4213701215 learned_addr learned my addr 172.21.15.112:0/4213701215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f14177fe640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1418102af0 0x7f141819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1416ffd640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1418108a20 0x7f141819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 -- 172.21.15.112:0/4213701215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1418108a20 msgr2=0x7f141819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1418108a20 0x7f141819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 -- 172.21.15.112:0/4213701215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1418102af0 msgr2=0x7f141819b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1418102af0 0x7f141819b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.943+0000 7f1417fff640 1 -- 172.21.15.112:0/4213701215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14181a6910 con 0x7f141810d380 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.944+0000 7f1417fff640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 0x7f141819c080 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f141000efd0 tx=0x7f141000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.944+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410003b70 con 0x7f141810d380 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.944+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14181a6c00 con 0x7f141810d380 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.944+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1410003d10 con 0x7f141810d380 2024-04-25T16:54:48.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.944+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14100108a0 con 0x7f141810d380 2024-04-25T16:54:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.944+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a70d0 con 0x7f141810d380 2024-04-25T16:54:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.945+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141806afa0 con 0x7f141810d380 2024-04-25T16:54:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.946+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f1410004020 con 0x7f141810d380 2024-04-25T16:54:48.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.947+0000 7f1414ff9640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f14000afc60 0x7f14000b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.947+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8814+0+0 (secure 0 0 0) 0x7f14100df550 con 0x7f141810d380 2024-04-25T16:54:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.947+0000 7f14177fe640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f14000afc60 0x7f14000b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:48.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.947+0000 7f14177fe640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f14000afc60 0x7f14000b2170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1418103a20 tx=0x7f140c005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:48.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:48.952+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14100a7cd0 con 0x7f141810d380 2024-04-25T16:54:49.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a[29118]: 2024-04-25T16:54:48.812+0000 7f97aefd1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T16:54:49.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.176+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1418106950 con 0x7f141810d380 2024-04-25T16:54:49.198 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c[63451]: ignoring --setuser ceph since I am not root 2024-04-25T16:54:49.199 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c[63451]: ignoring --setgroup ceph since I am not root 2024-04-25T16:54:49.199 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-mds[63455]: -- 172.21.15.133:0/619363644 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558dd7de1a40 con 0x558dd7e29180 2024-04-25T16:54:49.199 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c[63451]: starting mds.c at 2024-04-25T16:54:49.199 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i[63759]: ignoring --setuser ceph since I am not root 2024-04-25T16:54:49.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i[63759]: ignoring --setgroup ceph since I am not root 2024-04-25T16:54:49.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-mds[63763]: -- 172.21.15.133:0/1182638182 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b3ab2aba40 con 0x55b3ab2f3600 2024-04-25T16:54:49.200 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i[63759]: starting mds.i at 2024-04-25T16:54:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l[62835]: ignoring --setuser ceph since I am not root 2024-04-25T16:54:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l[62835]: ignoring --setgroup ceph since I am not root 2024-04-25T16:54:49.200 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-mds[62839]: -- 172.21.15.133:0/2339805136 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634f8727a40 con 0x5634f876f600 2024-04-25T16:54:49.201 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l[62835]: starting mds.l at 2024-04-25T16:54:49.201 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f[63140]: ignoring --setuser ceph since I am not root 2024-04-25T16:54:49.201 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f[63140]: ignoring --setgroup ceph since I am not root 2024-04-25T16:54:49.201 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-mds[63144]: -- 172.21.15.133:0/2590600770 <== mon.1 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ccd9a7da40 con 0x55ccd9ac5180 2024-04-25T16:54:49.201 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:54:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f[63140]: starting mds.f at 2024-04-25T16:54:49.257 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h[63069]: ignoring --setuser ceph since I am not root 2024-04-25T16:54:49.257 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h[63069]: ignoring --setgroup ceph since I am not root 2024-04-25T16:54:49.258 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-mds[63073]: -- 172.21.15.116:0/2970418168 <== mon.2 v2:172.21.15.116:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5ea3c3a40 con 0x55b5ea40b180 2024-04-25T16:54:49.258 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h[63069]: starting mds.h at 2024-04-25T16:54:49.258 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: ignoring --setuser ceph since I am not root 2024-04-25T16:54:49.258 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: ignoring --setgroup ceph since I am not root 2024-04-25T16:54:49.258 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-mds[63387]: -- 172.21.15.116:0/3323457350 <== mon.0 v2:172.21.15.112:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ffd335a40 con 0x555ffd37d180 2024-04-25T16:54:49.259 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:54:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: starting mds.b at 2024-04-25T16:54:49.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: pgmap v522: 97 pgs: 97 active+clean; 300 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 151 KiB/s wr, 17 op/s 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/3258810276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/553605277' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/553605277' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T16:54:49.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T16:54:49.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:49 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4213701215' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:54:49.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: pgmap v522: 97 pgs: 97 active+clean; 300 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 151 KiB/s wr, 17 op/s 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/3258810276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/553605277' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/553605277' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T16:54:49.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:49 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4213701215' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:54:49.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.816+0000 7f1414ff9640 1 -- 172.21.15.112:0/4213701215 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) ==== 104+0+0 (secure 0 0 0) 0x7f14100a75d0 con 0x7f141810d380 2024-04-25T16:54:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f14000afc60 msgr2=0x7f14000b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f14000afc60 0x7f14000b2170 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1418103a20 tx=0x7f140c005f90 comp rx=0 tx=0).stop 2024-04-25T16:54:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 msgr2=0x7f141819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 0x7f141819c080 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f141000efd0 tx=0x7f141000c6a0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 shutdown_connections 2024-04-25T16:54:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f14000afc60 0x7f14000b2170 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1418108a20 0x7f141819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f141810d380 0x7f141819c080 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 --2- 172.21.15.112:0/4213701215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1418102af0 0x7f141819b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 >> 172.21.15.112:0/4213701215 conn(0x7f1418078100 msgr2=0x7f14181040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 shutdown_connections 2024-04-25T16:54:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.818+0000 7f141df72640 1 -- 172.21.15.112:0/4213701215 wait complete. 2024-04-25T16:54:49.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: pgmap v522: 97 pgs: 97 active+clean; 300 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 151 KiB/s wr, 17 op/s 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/3258810276' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/553605277' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/553605277' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T16:54:49.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:49 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4213701215' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T16:54:49.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T16:54:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 -- 172.21.15.112:0/2182915182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc102af0 msgr2=0x7f8bfc102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 --2- 172.21.15.112:0/2182915182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc102af0 0x7f8bfc102f00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009900 tx=0x7f8bec02a410 comp rx=0 tx=0).stop 2024-04-25T16:54:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 -- 172.21.15.112:0/2182915182 shutdown_connections 2024-04-25T16:54:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 --2- 172.21.15.112:0/2182915182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bfc10d380 0x7f8bfc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 --2- 172.21.15.112:0/2182915182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc102af0 0x7f8bfc102f00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 --2- 172.21.15.112:0/2182915182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bfc108a20 0x7f8bfc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 -- 172.21.15.112:0/2182915182 >> 172.21.15.112:0/2182915182 conn(0x7f8bfc078100 msgr2=0x7f8bfc078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 -- 172.21.15.112:0/2182915182 shutdown_connections 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.947+0000 7f8c03384640 1 -- 172.21.15.112:0/2182915182 wait complete. 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 Processor -- start 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 -- start start 2024-04-25T16:54:49.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bfc102af0 0x7f8bfc1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bfc108a20 0x7f8bfc1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 0x7f8bfc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc112be0 con 0x7f8bfc10d380 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc112a60 con 0x7f8bfc102af0 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c03384640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc112d60 con 0x7f8bfc108a20 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 0x7f8bfc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 0x7f8bfc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56416/0 (socket says 172.21.15.112:56416) 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 -- 172.21.15.112:0/2769935915 learned_addr learned my addr 172.21.15.112:0/2769935915 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:49.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c0091f640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bfc108a20 0x7f8bfc1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01120640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bfc102af0 0x7f8bfc1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 -- 172.21.15.112:0/2769935915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bfc108a20 msgr2=0x7f8bfc1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bfc108a20 0x7f8bfc1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 -- 172.21.15.112:0/2769935915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bfc102af0 msgr2=0x7f8bfc1a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bfc102af0 0x7f8bfc1a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.948+0000 7f8c01921640 1 -- 172.21.15.112:0/2769935915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc19ada0 con 0x7f8bfc10d380 2024-04-25T16:54:49.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.949+0000 7f8c01921640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 0x7f8bfc19a670 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400eaa0 tx=0x7f8bf400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.949+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400cdb0 con 0x7f8bfc10d380 2024-04-25T16:54:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.949+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf4005020 con 0x7f8bfc10d380 2024-04-25T16:54:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.949+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40116d0 con 0x7f8bfc10d380 2024-04-25T16:54:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.949+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc19b030 con 0x7f8bfc10d380 2024-04-25T16:54:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.949+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1a7290 con 0x7f8bfc10d380 2024-04-25T16:54:49.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.950+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bfc103cd0 con 0x7f8bfc10d380 2024-04-25T16:54:49.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.951+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7f8bf40051d0 con 0x7f8bfc10d380 2024-04-25T16:54:49.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.952+0000 7f8bf27fc640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8be80afce0 0x7f8be80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:49.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.952+0000 7f8c01120640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8be80afce0 0x7f8be80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:49.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.952+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8814+0+0 (secure 0 0 0) 0x7f8bf40df6f0 con 0x7f8bfc10d380 2024-04-25T16:54:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.952+0000 7f8c01120640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8be80afce0 0x7f8be80b21f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005fd0 tx=0x7f8bf8005850 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:49.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:49.956+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8bf4016030 con 0x7f8bfc10d380 2024-04-25T16:54:50.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.134+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8bfc1a7460 con 0x7f8bfc10d380 2024-04-25T16:54:50.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: Cluster is now healthy 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/4213701215' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6832/2843247771,v1:172.21.15.133:6835/2843247771] up:boot 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6833/2183188601,v1:172.21.15.133:6834/2183188601] up:boot 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6837/3181250821,v1:172.21.15.116:6839/3181250821] up:boot 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.116:6836/2855223649,v1:172.21.15.116:6838/2855223649] up:boot 2024-04-25T16:54:50.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6836/3017652444,v1:172.21.15.133:6838/3017652444] up:boot 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: mds.? [v2:172.21.15.133:6837/482016961,v1:172.21.15.133:6839/482016961] up:boot 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: fsmap 12 up:standby 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:54:50.700 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:50 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2769935915' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T16:54:50.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: Cluster is now healthy 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/4213701215' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6832/2843247771,v1:172.21.15.133:6835/2843247771] up:boot 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6833/2183188601,v1:172.21.15.133:6834/2183188601] up:boot 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6837/3181250821,v1:172.21.15.116:6839/3181250821] up:boot 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.116:6836/2855223649,v1:172.21.15.116:6838/2855223649] up:boot 2024-04-25T16:54:50.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6836/3017652444,v1:172.21.15.133:6838/3017652444] up:boot 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: mds.? [v2:172.21.15.133:6837/482016961,v1:172.21.15.133:6839/482016961] up:boot 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: fsmap 12 up:standby 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:54:50.759 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:50 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2769935915' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T16:54:50.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.817+0000 7f8bf27fc640 1 -- 172.21.15.112:0/2769935915 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7f8bf40a7ef0 con 0x7f8bfc10d380 2024-04-25T16:54:50.815 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-04-25T16:54:50.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8be80afce0 msgr2=0x7f8be80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:50.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8be80afce0 0x7f8be80b21f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8005fd0 tx=0x7f8bf8005850 comp rx=0 tx=0).stop 2024-04-25T16:54:50.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 msgr2=0x7f8bfc19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:50.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 0x7f8bfc19a670 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400eaa0 tx=0x7f8bf400ef60 comp rx=0 tx=0).stop 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 shutdown_connections 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7f8be80afce0 0x7f8be80b21f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8bfc108a20 0x7f8bfc1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bfc10d380 0x7f8bfc19a670 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 --2- 172.21.15.112:0/2769935915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bfc102af0 0x7f8bfc1a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 >> 172.21.15.112:0/2769935915 conn(0x7f8bfc078100 msgr2=0x7f8bfc069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 shutdown_connections 2024-04-25T16:54:50.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.819+0000 7f8c03384640 1 -- 172.21.15.112:0/2769935915 wait complete. 2024-04-25T16:54:50.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T16:54:50.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T16:54:50.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: Cluster is now healthy 2024-04-25T16:54:50.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/4213701215' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T16:54:50.827 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6832/2843247771,v1:172.21.15.133:6835/2843247771] up:boot 2024-04-25T16:54:50.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6833/2183188601,v1:172.21.15.133:6834/2183188601] up:boot 2024-04-25T16:54:50.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6837/3181250821,v1:172.21.15.116:6839/3181250821] up:boot 2024-04-25T16:54:50.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.116:6836/2855223649,v1:172.21.15.116:6838/2855223649] up:boot 2024-04-25T16:54:50.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6836/3017652444,v1:172.21.15.133:6838/3017652444] up:boot 2024-04-25T16:54:50.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: mds.? [v2:172.21.15.133:6837/482016961,v1:172.21.15.133:6839/482016961] up:boot 2024-04-25T16:54:50.828 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: fsmap 12 up:standby 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T16:54:50.829 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:50 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2769935915' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T16:54:50.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T16:54:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.946+0000 7fabe3ae0640 1 -- 172.21.15.112:0/1732790304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108e00 msgr2=0x7fabdc110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.946+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/1732790304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108e00 0x7fabdc110630 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009ca0 tx=0x7fabd402a8c0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 -- 172.21.15.112:0/1732790304 shutdown_connections 2024-04-25T16:54:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/1732790304 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108e00 0x7fabdc110630 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/1732790304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc108380 0x7fabdc108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/1732790304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc102210 0x7fabdc102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 -- 172.21.15.112:0/1732790304 >> 172.21.15.112:0/1732790304 conn(0x7fabdc077880 msgr2=0x7fabdc077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:50.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 -- 172.21.15.112:0/1732790304 shutdown_connections 2024-04-25T16:54:50.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 -- 172.21.15.112:0/1732790304 wait complete. 2024-04-25T16:54:50.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 Processor -- start 2024-04-25T16:54:50.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.947+0000 7fabe3ae0640 1 -- start start 2024-04-25T16:54:50.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc102210 0x7fabdc19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:50.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 0x7fabdc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc108e00 0x7fabdc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc113d20 con 0x7fabdc108380 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc113ba0 con 0x7fabdc102210 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fabdc113ea0 con 0x7fabdc108e00 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 0x7fabdc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 0x7fabdc19aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:56430/0 (socket says 172.21.15.112:56430) 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 -- 172.21.15.112:0/2994228627 learned_addr learned my addr 172.21.15.112:0/2994228627 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe187c640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc102210 0x7fabdc19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:50.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe207d640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc108e00 0x7fabdc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 -- 172.21.15.112:0/2994228627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc108e00 msgr2=0x7fabdc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc108e00 0x7fabdc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 -- 172.21.15.112:0/2994228627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc102210 msgr2=0x7fabdc19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc102210 0x7fabdc19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 -- 172.21.15.112:0/2994228627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabdc19f4a0 con 0x7fabdc108380 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe187c640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc102210 0x7fabdc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe107b640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 0x7fabdc19aba0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00e9f0 tx=0x7fabcc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:50.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00ccc0 con 0x7fabdc108380 2024-04-25T16:54:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabcc0035f0 con 0x7fabdc108380 2024-04-25T16:54:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabdc19f790 con 0x7fabdc108380 2024-04-25T16:54:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabcc00ce60 con 0x7fabdc108380 2024-04-25T16:54:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.948+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabdc1a7110 con 0x7fabdc108380 2024-04-25T16:54:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.950+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabdc06a190 con 0x7fabdc108380 2024-04-25T16:54:50.951 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.951+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 25) ==== 147652+0+0 (secure 0 0 0) 0x7fabcc003aa0 con 0x7fabdc108380 2024-04-25T16:54:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.952+0000 7fabd2ffd640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fabc80afce0 0x7fabc80b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:54:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.952+0000 7fabe187c640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fabc80afce0 0x7fabc80b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:54:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.952+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8289+0+0 (secure 0 0 0) 0x7fabcc0df1d0 con 0x7fabdc108380 2024-04-25T16:54:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.952+0000 7fabe187c640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fabc80afce0 0x7fabc80b21f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fabdc06b550 tx=0x7fabd8005f90 comp rx=0 tx=0).ready entity=mgr.15294 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:54:50.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:50.957+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabcc0a7b50 con 0x7fabdc108380 2024-04-25T16:54:51.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.170+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fabdc10bc60 con 0x7fabdc108380 2024-04-25T16:54:51.698 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: pgmap v524: 97 pgs: 97 active+clean; 301 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 156 KiB/s wr, 12 op/s 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2769935915' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: from='client.? 172.21.15.112:0/2994228627' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:54:51.699 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:51 smithi133 ceph-mon[32691]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:54:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: pgmap v524: 97 pgs: 97 active+clean; 301 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 156 KiB/s wr, 12 op/s 2024-04-25T16:54:51.757 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:54:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2769935915' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T16:54:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T16:54:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: from='client.? 172.21.15.112:0/2994228627' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T16:54:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:54:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:54:51.758 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:51 smithi116 ceph-mon[32712]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:54:51.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.818+0000 7fabd2ffd640 1 -- 172.21.15.112:0/2994228627 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7fabcc0a7450 con 0x7fabdc108380 2024-04-25T16:54:51.816 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-04-25T16:54:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fabc80afce0 msgr2=0x7fabc80b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fabc80afce0 0x7fabc80b21f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fabdc06b550 tx=0x7fabd8005f90 comp rx=0 tx=0).stop 2024-04-25T16:54:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 msgr2=0x7fabdc19aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:54:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 0x7fabdc19aba0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00e9f0 tx=0x7fabcc00eeb0 comp rx=0 tx=0).stop 2024-04-25T16:54:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 shutdown_connections 2024-04-25T16:54:51.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:6800/779201591,v1:172.21.15.112:6801/779201591] conn(0x7fabc80afce0 0x7fabc80b21f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:51.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fabdc108e00 0x7fabdc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:51.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fabdc108380 0x7fabdc19aba0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:51.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 --2- 172.21.15.112:0/2994228627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabdc102210 0x7fabdc19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:54:51.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 >> 172.21.15.112:0/2994228627 conn(0x7fabdc077880 msgr2=0x7fabdc0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:54:51.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 shutdown_connections 2024-04-25T16:54:51.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T16:54:51.821+0000 7fabe3ae0640 1 -- 172.21.15.112:0/2994228627 wait complete. 2024-04-25T16:54:51.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: pgmap v524: 97 pgs: 97 active+clean; 301 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 156 KiB/s wr, 12 op/s 2024-04-25T16:54:51.832 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:54:51.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2769935915' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T16:54:51.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: osdmap e111: 12 total, 12 up, 12 in 2024-04-25T16:54:51.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: from='client.? 172.21.15.112:0/2994228627' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T16:54:51.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:54:51.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:54:51.833 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:51 smithi112 ceph-mon[29122]: from='mgr.15294 172.21.15.112:0/3273592137' entity='mgr.x' 2024-04-25T16:54:51.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:54:51.870 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:54:51.909 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:54:51.942 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T16:54:51.942 DEBUG:teuthology.orchestra.run.smithi112:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T16:54:51.958 DEBUG:teuthology.orchestra.run.smithi116:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T16:54:51.974 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T16:54:51.999 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T16:54:52.000 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T16:54:52.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a 2024-04-25T16:54:52.094 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:52 smithi112 systemd[1]: Stopping Ceph mon.a for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:54:52.694 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a[29118]: 2024-04-25T16:54:52.431+0000 7f97b47dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:54:52.694 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a[29118]: 2024-04-25T16:54:52.431+0000 7f97b47dc700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T16:54:53.003 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:52 smithi112 podman[91913]: 2024-04-25 16:54:52.697618566 +0000 UTC m=+0.403114826 container died 60a3b4a63620d25f1ecc2ae158fee36f7bc4c3fbe31fb8fa93f6e368ccc9c48b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T16:54:53.337 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:53 smithi112 podman[91913]: 2024-04-25 16:54:53.105640758 +0000 UTC m=+0.811137023 container cleanup 60a3b4a63620d25f1ecc2ae158fee36f7bc4c3fbe31fb8fa93f6e368ccc9c48b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T16:54:53.338 INFO:journalctl@ceph.mon.a.smithi112.stdout:Apr 25 16:54:53 smithi112 bash[91913]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-a 2024-04-25T16:54:53.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T16:54:53.505 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:54:53.505 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T16:54:53.506 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T16:54:53.506 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.b 2024-04-25T16:54:53.864 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:53 smithi116 systemd[1]: Stopping Ceph mon.b for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:54:54.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b[32707]: 2024-04-25T16:54:53.866+0000 7f87d203a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:54:54.257 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b[32707]: 2024-04-25T16:54:53.866+0000 7f87d203a700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T16:54:54.258 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:53 smithi116 podman[70524]: 2024-04-25 16:54:53.968485634 +0000 UTC m=+0.227713438 container died dd884f0c9137ae8dff8c5059cfe190126d52291a69fb06d309f8198626570551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T16:54:54.606 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:54 smithi116 podman[70524]: 2024-04-25 16:54:54.375128854 +0000 UTC m=+0.634356667 container cleanup dd884f0c9137ae8dff8c5059cfe190126d52291a69fb06d309f8198626570551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T16:54:54.607 INFO:journalctl@ceph.mon.b.smithi116.stdout:Apr 25 16:54:54 smithi116 bash[70524]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-b 2024-04-25T16:54:54.823 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T16:54:54.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:54:54.867 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T16:54:54.868 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T16:54:54.868 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.c 2024-04-25T16:54:55.189 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:54 smithi133 systemd[1]: Stopping Ceph mon.c for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:54:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-c[32683]: 2024-04-25T16:54:55.190+0000 7f69b030d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:54:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-c[32683]: 2024-04-25T16:54:55.190+0000 7f69b030d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T16:54:55.449 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:55 smithi133 podman[70290]: 2024-04-25 16:54:55.212622882 +0000 UTC m=+0.148122711 container died 144bc0b8c319c852413e89c3e67d274354e64fbeea906c582dd86cb7d66b6dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T16:54:55.917 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:55 smithi133 podman[70290]: 2024-04-25 16:54:55.601347202 +0000 UTC m=+0.536847039 container cleanup 144bc0b8c319c852413e89c3e67d274354e64fbeea906c582dd86cb7d66b6dd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T16:54:55.917 INFO:journalctl@ceph.mon.c.smithi133.stdout:Apr 25 16:54:55 smithi133 bash[70290]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mon-c 2024-04-25T16:54:56.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T16:54:56.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:54:56.119 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T16:54:56.120 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T16:54:56.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x 2024-04-25T16:54:56.416 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:54:56 smithi112 systemd[1]: Stopping Ceph mgr.x for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:54:56.751 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:54:56 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x[29351]: 2024-04-25T16:54:56.418+0000 7fb5bb449700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:54:56.751 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:54:56 smithi112 podman[92035]: 2024-04-25 16:54:56.449993593 +0000 UTC m=+0.133798433 container died d8c4e38aecbcb3ee5e0c99e4c1ee0acb6c71d5f44bf19ae5044ea5ea2c47cad2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:54:57.085 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:54:56 smithi112 podman[92035]: 2024-04-25 16:54:56.853050921 +0000 UTC m=+0.536855761 container cleanup d8c4e38aecbcb3ee5e0c99e4c1ee0acb6c71d5f44bf19ae5044ea5ea2c47cad2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T16:54:57.085 INFO:journalctl@ceph.mgr.x.smithi112.stdout:Apr 25 16:54:56 smithi112 bash[92035]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-x 2024-04-25T16:54:57.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T16:54:57.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:54:57.265 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T16:54:57.266 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T16:54:57.266 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.y 2024-04-25T16:54:57.624 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:54:57 smithi116 systemd[1]: Stopping Ceph mgr.y for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:54:57.947 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:54:57 smithi116 podman[70644]: 2024-04-25 16:54:57.627425548 +0000 UTC m=+0.156597417 container died 77c04953f41c5bbbf0a7b8f83ad2d0f6bb8cbc57ac78f502df0574d9dd8cdddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:54:58.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:54:58 smithi116 podman[70644]: 2024-04-25 16:54:58.056834682 +0000 UTC m=+0.586006550 container cleanup 77c04953f41c5bbbf0a7b8f83ad2d0f6bb8cbc57ac78f502df0574d9dd8cdddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T16:54:58.258 INFO:journalctl@ceph.mgr.y.smithi116.stdout:Apr 25 16:54:58 smithi116 bash[70644]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-y 2024-04-25T16:54:58.544 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T16:54:58.577 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:54:58.577 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T16:54:58.577 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T16:54:58.577 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.z 2024-04-25T16:54:58.909 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:54:58 smithi133 systemd[1]: Stopping Ceph mgr.z for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:54:59.198 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:54:58 smithi133 podman[70414]: 2024-04-25 16:54:58.912075456 +0000 UTC m=+0.138120264 container died b0b3a1821e60261d6092dbe13ecddb7a92607f16c7b998eda47a39112d5091a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T16:54:59.556 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:54:59 smithi133 podman[70414]: 2024-04-25 16:54:59.298573158 +0000 UTC m=+0.524617945 container cleanup b0b3a1821e60261d6092dbe13ecddb7a92607f16c7b998eda47a39112d5091a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:54:59.556 INFO:journalctl@ceph.mgr.z.smithi133.stdout:Apr 25 16:54:59 smithi133 bash[70414]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mgr-z 2024-04-25T16:54:59.709 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T16:54:59.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:54:59.744 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T16:54:59.744 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T16:54:59.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.0 2024-04-25T16:55:00.072 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:54:59 smithi112 systemd[1]: Stopping Ceph osd.0 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:00.344 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0[44345]: 2024-04-25T16:55:00.074+0000 7fb3610f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:00.345 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0[44345]: 2024-04-25T16:55:00.074+0000 7fb3610f4700 -1 osd.0 112 *** Got signal Terminated *** 2024-04-25T16:55:00.345 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0[44345]: 2024-04-25T16:55:00.074+0000 7fb3610f4700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:05.845 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:05 smithi112 podman[92161]: 2024-04-25 16:55:05.456236196 +0000 UTC m=+5.507358053 container died 13a0259ea1b50a90a3c9506647ac57003cb4a52e2f1fa7bf0403caa51714c472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:55:06.182 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:05 smithi112 podman[92161]: 2024-04-25 16:55:05.967737114 +0000 UTC m=+6.018858980 container cleanup 13a0259ea1b50a90a3c9506647ac57003cb4a52e2f1fa7bf0403caa51714c472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T16:55:06.182 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:05 smithi112 bash[92161]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0 2024-04-25T16:55:06.501 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:06 smithi112 podman[92180]: 2024-04-25 16:55:06.185798185 +0000 UTC m=+0.726481950 container remove 13a0259ea1b50a90a3c9506647ac57003cb4a52e2f1fa7bf0403caa51714c472 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T16:55:06.828 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:06 smithi112 podman[92266]: 2024-04-25 16:55:06.408351701 +0000 UTC m=+0.018209283 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:06.828 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:06 smithi112 podman[92266]: 2024-04-25 16:55:06.528410972 +0000 UTC m=+0.138268553 container create 05bec2e5a2452ca207153e018e1094f24730b24b16898df0ba7d0ffb8cdd7809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:06.828 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:06 smithi112 podman[92266]: 2024-04-25 16:55:06.739467774 +0000 UTC m=+0.349325355 container init 05bec2e5a2452ca207153e018e1094f24730b24b16898df0ba7d0ffb8cdd7809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:55:06.828 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:06 smithi112 podman[92266]: 2024-04-25 16:55:06.742574295 +0000 UTC m=+0.352431882 container start 05bec2e5a2452ca207153e018e1094f24730b24b16898df0ba7d0ffb8cdd7809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T16:55:06.828 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:06 smithi112 podman[92266]: 2024-04-25 16:55:06.831219004 +0000 UTC m=+0.441076596 container attach 05bec2e5a2452ca207153e018e1094f24730b24b16898df0ba7d0ffb8cdd7809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:55:07.094 INFO:journalctl@ceph.osd.0.smithi112.stdout:Apr 25 16:55:07 smithi112 podman[92266]: 2024-04-25 16:55:07.04168432 +0000 UTC m=+0.651541895 container died 05bec2e5a2452ca207153e018e1094f24730b24b16898df0ba7d0ffb8cdd7809 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T16:55:08.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T16:55:08.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:08.495 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T16:55:08.495 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T16:55:08.495 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.1 2024-04-25T16:55:08.810 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:08 smithi116 systemd[1]: Stopping Ceph osd.1 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:09.257 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1[37293]: 2024-04-25T16:55:08.812+0000 7fe6b794f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:09.257 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1[37293]: 2024-04-25T16:55:08.812+0000 7fe6b794f700 -1 osd.1 112 *** Got signal Terminated *** 2024-04-25T16:55:09.258 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1[37293]: 2024-04-25T16:55:08.812+0000 7fe6b794f700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:14.507 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:14 smithi116 podman[70779]: 2024-04-25 16:55:14.256926057 +0000 UTC m=+5.565229015 container died f850961f5128b93378cab9eb0f2c6dfd6a9fa115650159f7a74798b151651d6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T16:55:14.927 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:14 smithi116 podman[70779]: 2024-04-25 16:55:14.646051848 +0000 UTC m=+5.954354794 container cleanup f850961f5128b93378cab9eb0f2c6dfd6a9fa115650159f7a74798b151651d6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T16:55:14.928 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:14 smithi116 bash[70779]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1 2024-04-25T16:55:15.231 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:14 smithi116 podman[70796]: 2024-04-25 16:55:14.931021206 +0000 UTC m=+0.668126273 container remove f850961f5128b93378cab9eb0f2c6dfd6a9fa115650159f7a74798b151651d6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T16:55:15.507 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:15 smithi116 podman[70884]: 2024-04-25 16:55:15.136906598 +0000 UTC m=+0.017768558 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:15.508 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:15 smithi116 podman[70884]: 2024-04-25 16:55:15.2570542 +0000 UTC m=+0.137916147 container create 1a2355586c00f0f85adebdbfcea7b7d6d64ac2156a411a1dee4dc1ec6910f803 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T16:55:15.758 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:15 smithi116 podman[70884]: 2024-04-25 16:55:15.510591292 +0000 UTC m=+0.391453248 container init 1a2355586c00f0f85adebdbfcea7b7d6d64ac2156a411a1dee4dc1ec6910f803 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T16:55:15.758 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:15 smithi116 podman[70884]: 2024-04-25 16:55:15.514347838 +0000 UTC m=+0.395209792 container start 1a2355586c00f0f85adebdbfcea7b7d6d64ac2156a411a1dee4dc1ec6910f803 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T16:55:15.758 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:15 smithi116 podman[70884]: 2024-04-25 16:55:15.643511048 +0000 UTC m=+0.524373015 container attach 1a2355586c00f0f85adebdbfcea7b7d6d64ac2156a411a1dee4dc1ec6910f803 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T16:55:16.507 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:16 smithi116 podman[70884]: 2024-04-25 16:55:16.07119754 +0000 UTC m=+0.952059489 container died 1a2355586c00f0f85adebdbfcea7b7d6d64ac2156a411a1dee4dc1ec6910f803 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T16:55:16.876 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T16:55:16.876 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:16 smithi116 podman[70910]: 2024-04-25 16:55:16.856626642 +0000 UTC m=+0.780194619 container remove 1a2355586c00f0f85adebdbfcea7b7d6d64ac2156a411a1dee4dc1ec6910f803 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:16.876 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:16 smithi116 systemd[1]: ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-25T16:55:16.877 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:16 smithi116 systemd[1]: Stopped Ceph osd.1 for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:55:16.877 INFO:journalctl@ceph.osd.1.smithi116.stdout:Apr 25 16:55:16 smithi116 systemd[1]: ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.1.service: Consumed 3min 12.668s CPU time. 2024-04-25T16:55:16.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:16.917 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T16:55:16.917 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T16:55:16.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.2 2024-04-25T16:55:17.199 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:16 smithi133 systemd[1]: Stopping Ceph osd.2 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:17.698 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2[37866]: 2024-04-25T16:55:17.252+0000 7fbee03cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:17.699 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2[37866]: 2024-04-25T16:55:17.252+0000 7fbee03cc700 -1 osd.2 112 *** Got signal Terminated *** 2024-04-25T16:55:17.699 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2[37866]: 2024-04-25T16:55:17.252+0000 7fbee03cc700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:22.949 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:22 smithi133 podman[70551]: 2024-04-25 16:55:22.673910301 +0000 UTC m=+5.542707382 container died f833ac8eb8602089c0fa30e4df4ef746217f428a9f3d2185bb71b3c7a8479a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-25T16:55:23.310 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70551]: 2024-04-25 16:55:23.036261734 +0000 UTC m=+5.905058804 container cleanup f833ac8eb8602089c0fa30e4df4ef746217f428a9f3d2185bb71b3c7a8479a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T16:55:23.310 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 bash[70551]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2 2024-04-25T16:55:23.609 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70568]: 2024-04-25 16:55:23.313101942 +0000 UTC m=+0.638596230 container remove f833ac8eb8602089c0fa30e4df4ef746217f428a9f3d2185bb71b3c7a8479a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux ) 2024-04-25T16:55:23.880 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70658]: 2024-04-25 16:55:23.515004036 +0000 UTC m=+0.017883262 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:23.880 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70658]: 2024-04-25 16:55:23.647358855 +0000 UTC m=+0.150238065 container create ad803cdbdac9319bd8ec01d90a535d2a69282529594290b3a49520fba6bcfed6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:55:23.880 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70658]: 2024-04-25 16:55:23.883546984 +0000 UTC m=+0.386426194 container init ad803cdbdac9319bd8ec01d90a535d2a69282529594290b3a49520fba6bcfed6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T16:55:24.175 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70658]: 2024-04-25 16:55:23.886972602 +0000 UTC m=+0.389851813 container start ad803cdbdac9319bd8ec01d90a535d2a69282529594290b3a49520fba6bcfed6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T16:55:24.175 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:23 smithi133 podman[70658]: 2024-04-25 16:55:23.966945674 +0000 UTC m=+0.469824900 container attach ad803cdbdac9319bd8ec01d90a535d2a69282529594290b3a49520fba6bcfed6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T16:55:24.448 INFO:journalctl@ceph.osd.2.smithi133.stdout:Apr 25 16:55:24 smithi133 podman[70658]: 2024-04-25 16:55:24.177965113 +0000 UTC m=+0.680844327 container died ad803cdbdac9319bd8ec01d90a535d2a69282529594290b3a49520fba6bcfed6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:24.857 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T16:55:24.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:24.895 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T16:55:24.896 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T16:55:24.896 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.3 2024-04-25T16:55:25.194 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:24 smithi112 systemd[1]: Stopping Ceph osd.3 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:25.486 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:25.196+0000 7f69c2ee1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:25.487 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:25.196+0000 7f69c2ee1700 -1 osd.3 112 *** Got signal Terminated *** 2024-04-25T16:55:25.487 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:25.196+0000 7f69c2ee1700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:25.844 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:25 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:25.684+0000 7f69bf497700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.536788+0000 front 2024-04-25T16:55:04.536830+0000 (oldest deadline 2024-04-25T16:55:25.637015+0000) 2024-04-25T16:55:26.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:26.476+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:27.094 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:26 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:26.704+0000 7f69bf497700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.536788+0000 front 2024-04-25T16:55:04.536830+0000 (oldest deadline 2024-04-25T16:55:25.637015+0000) 2024-04-25T16:55:27.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:26.869+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:27.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:27.481+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:27.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:27.317+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:27.719 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:27.408+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:28.094 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:27.721+0000 7f69bf497700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.536788+0000 front 2024-04-25T16:55:04.536830+0000 (oldest deadline 2024-04-25T16:55:25.637015+0000) 2024-04-25T16:55:28.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:27 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:27.746+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:28.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:27 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:27.866+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:28.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:28.287+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:28.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:28.439+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:28.763 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:28.705+0000 7f69bf497700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.536788+0000 front 2024-04-25T16:55:04.536830+0000 (oldest deadline 2024-04-25T16:55:25.637015+0000) 2024-04-25T16:55:28.763 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:28.457+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:29.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:28 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:28.766+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:29.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:28 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:28.858+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:29.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:29.242+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:29.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:29.407+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:29.725 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:29 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:29.438+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:30.094 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:29 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3[51200]: 2024-04-25T16:55:29.727+0000 7f69bf497700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.536788+0000 front 2024-04-25T16:55:04.536830+0000 (oldest deadline 2024-04-25T16:55:25.637015+0000) 2024-04-25T16:55:30.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:29 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:29.747+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:30.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:29 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:29.890+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:30.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:30.419+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:30.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:30.292+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:30.763 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:30 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:30.429+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:31.053 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:30 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:30.765+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:31.054 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:30 smithi112 podman[92374]: 2024-04-25 16:55:30.770797694 +0000 UTC m=+5.679549396 container died 9955c603034d9a0141394d1e890a0da191805901ba97eb35b84a88816d63e9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T16:55:31.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:30 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:30.841+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:31.344 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:31 smithi112 podman[92374]: 2024-04-25 16:55:31.155665389 +0000 UTC m=+6.064417097 container cleanup 9955c603034d9a0141394d1e890a0da191805901ba97eb35b84a88816d63e9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:55:31.345 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:31 smithi112 bash[92374]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3 2024-04-25T16:55:31.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:31.439+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:31.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:31.311+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:31.779 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:31 smithi112 podman[92403]: 2024-04-25 16:55:31.415592408 +0000 UTC m=+0.643548600 container remove 9955c603034d9a0141394d1e890a0da191805901ba97eb35b84a88816d63e9a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T16:55:31.779 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:31.473+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:32.041 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:31 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:31.782+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:32.042 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:31 smithi112 podman[92499]: 2024-04-25 16:55:31.729079886 +0000 UTC m=+0.018431059 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:32.042 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:31 smithi112 podman[92499]: 2024-04-25 16:55:31.83334589 +0000 UTC m=+0.122697031 container create cc35b8998eea12c78b2e16ab4fa9be7955676806a07140d604faad933079b95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:32.042 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:32 smithi112 podman[92499]: 2024-04-25 16:55:32.044692569 +0000 UTC m=+0.334043710 container init cc35b8998eea12c78b2e16ab4fa9be7955676806a07140d604faad933079b95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-25T16:55:32.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:31 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:31.797+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:32.343 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:32 smithi112 podman[92499]: 2024-04-25 16:55:32.048040554 +0000 UTC m=+0.337391695 container start cc35b8998eea12c78b2e16ab4fa9be7955676806a07140d604faad933079b95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:32.343 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:32 smithi112 podman[92499]: 2024-04-25 16:55:32.119569794 +0000 UTC m=+0.408920938 container attach cc35b8998eea12c78b2e16ab4fa9be7955676806a07140d604faad933079b95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T16:55:32.594 INFO:journalctl@ceph.osd.3.smithi112.stdout:Apr 25 16:55:32 smithi112 podman[92499]: 2024-04-25 16:55:32.346392679 +0000 UTC m=+0.635743833 container died cc35b8998eea12c78b2e16ab4fa9be7955676806a07140d604faad933079b95f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T16:55:32.594 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:32 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:32.483+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:32.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:32.486+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:32.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:32.310+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:33.012 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:32 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:32.781+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:33.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T16:55:33.070 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:33.070 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T16:55:33.071 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T16:55:33.071 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.4 2024-04-25T16:55:33.097 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:32 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:32.788+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:33.382 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:33 smithi116 systemd[1]: Stopping Ceph osd.4 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:33.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:33 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:33.335+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:33.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:33 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:33.457+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:33.757 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:33 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4[44259]: 2024-04-25T16:55:33.384+0000 7fa5666e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:33.757 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:33 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4[44259]: 2024-04-25T16:55:33.384+0000 7fa5666e3700 -1 osd.4 112 *** Got signal Terminated *** 2024-04-25T16:55:33.758 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:33 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4[44259]: 2024-04-25T16:55:33.384+0000 7fa5666e3700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:33.797 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:33 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:33.517+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:34.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:33 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:33.800+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:34.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:33 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:33.834+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:34.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:34.317+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:34.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:34.418+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:34.827 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:34 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:34.500+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:35.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:34 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:34.830+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:35.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:34 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:34.824+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:35.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:34 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:34.824+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:35.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:35 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:35.307+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:35.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:35 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:35.425+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:35.753 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:35 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:35.472+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:36.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:35 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:35.843+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:36.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:35 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:35.783+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:36.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:35 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:35.783+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:36.594 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:36 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:36.447+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:36.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:36 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:36.466+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:36.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:36 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:36.466+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:36.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:36 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:36.261+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:36.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:36 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:36.261+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:37.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:36 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:36.805+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:37.158 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:36 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:36.761+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:37.159 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:36 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:36.761+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:37.600 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:37 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:37.465+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:37.600 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:37 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:37.465+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:37.600 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:37 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:37.288+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:37.600 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:37 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:37.288+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:37.822 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:37 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:37.403+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:38.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:37 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:37.781+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:38.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:37 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:37.781+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:38.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:37 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:37.572+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:38.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:37 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:37.825+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:38.616 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:38 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:38.442+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:38.617 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:38 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:38.442+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:38.617 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:38 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:38.325+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:38.617 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:38 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:38.325+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:38.772 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:38 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:38.429+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:38.772 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:38 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:38.429+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:38.831 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:38 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:38.583+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:38.831 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:38 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:38.748+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:38.831 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:38 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:38.748+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:39.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:38 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:38.775+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:39.122 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:38 smithi116 podman[70991]: 2024-04-25 16:55:38.834784123 +0000 UTC m=+5.571275352 container died 9d9111c54140a79404d7c3da50d4655ed24d23a6326d9ef073c8c20d869fe72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:55:39.507 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:39 smithi116 podman[70991]: 2024-04-25 16:55:39.23522938 +0000 UTC m=+5.971720616 container cleanup 9d9111c54140a79404d7c3da50d4655ed24d23a6326d9ef073c8c20d869fe72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T16:55:39.507 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:39 smithi116 bash[70991]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4 2024-04-25T16:55:39.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:39.478+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:39.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:39.478+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:39.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:39.301+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:39.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:39.301+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:39.699 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:39 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:39.251+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:39.742 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:39 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:39.402+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:39.742 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:39 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:39.402+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:39.857 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:39 smithi116 podman[71008]: 2024-04-25 16:55:39.536617955 +0000 UTC m=+0.695722275 container remove 9d9111c54140a79404d7c3da50d4655ed24d23a6326d9ef073c8c20d869fe72b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T16:55:39.858 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:39 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:39.578+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:39.858 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:39 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:39.719+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:39.858 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:39 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:39.719+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:40.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:39 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:39.745+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:40.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:39 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:39.745+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:40.204 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:39 smithi116 podman[71097]: 2024-04-25 16:55:39.763189977 +0000 UTC m=+0.018077268 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:40.204 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:39 smithi116 podman[71097]: 2024-04-25 16:55:39.862760543 +0000 UTC m=+0.117647818 container create b899a4517bc2cfe36b9e03eebedfef4fc34c9c7fd151ed7fb8f9be842aa087a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T16:55:40.204 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:40 smithi116 podman[71097]: 2024-04-25 16:55:40.107367196 +0000 UTC m=+0.362254472 container init b899a4517bc2cfe36b9e03eebedfef4fc34c9c7fd151ed7fb8f9be842aa087a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True) 2024-04-25T16:55:40.204 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:40 smithi116 podman[71097]: 2024-04-25 16:55:40.110541188 +0000 UTC m=+0.365428465 container start b899a4517bc2cfe36b9e03eebedfef4fc34c9c7fd151ed7fb8f9be842aa087a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:55:40.507 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:40 smithi116 podman[71097]: 2024-04-25 16:55:40.207269062 +0000 UTC m=+0.462156338 container attach b899a4517bc2cfe36b9e03eebedfef4fc34c9c7fd151ed7fb8f9be842aa087a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T16:55:40.507 INFO:journalctl@ceph.osd.4.smithi116.stdout:Apr 25 16:55:40 smithi116 podman[71097]: 2024-04-25 16:55:40.398525176 +0000 UTC m=+0.653412457 container died b899a4517bc2cfe36b9e03eebedfef4fc34c9c7fd151ed7fb8f9be842aa087a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:40.698 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:40 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:40.227+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:40.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:40 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:40.458+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:40.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:40 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:40.458+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:40.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:40 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:40.333+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:40.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:40 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:40.333+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:40.781 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:40 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:40.406+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:40.782 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:40 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:40.406+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:41.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:40 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:40.625+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:41.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:40 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:40.688+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:41.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:40 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:40.688+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:41.039 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T16:55:41.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:40 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:40.784+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:41.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:40 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:40.784+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:41.118 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:41.118 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T16:55:41.119 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T16:55:41.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.5 2024-04-25T16:55:41.437 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:41 smithi133 systemd[1]: Stopping Ceph osd.5 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:41.438 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:41.257+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:41.438 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:41.349+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:41.438 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:41.349+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:41.698 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:41.440+0000 7f23b8f00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:41.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:41.440+0000 7f23b8f00700 -1 osd.5 112 *** Got signal Terminated *** 2024-04-25T16:55:41.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:41.440+0000 7f23b8f00700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:41.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:41.486+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:41.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:41 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:41.486+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:41.827 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:41.437+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:41.828 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:41.437+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:42.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:41 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:41.597+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:42.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:41 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:41.648+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:42.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:41 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:41.648+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:42.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:41.830+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:42.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:41 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:41.830+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:42.515 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:42 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:42.266+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:42.516 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:42 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:42.329+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:42.516 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:42 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:42.329+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:42.802 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:42.478+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:42.803 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:42.478+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:42.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:42 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:42.518+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:42.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:42 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:42.518+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:43.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:42 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:42.570+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:43.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:42 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:42.627+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:43.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:42 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:42.627+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:43.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:42.805+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:43.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:42.805+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:43.495 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:43 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:43.248+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:43.496 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:43 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:43.293+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:43.496 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:43 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:43.293+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:43.811 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:43.476+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:43.812 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:43.476+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:43.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:43 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:43.498+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:43.949 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:43 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:43.498+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:43.949 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:43 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:43.498+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.556207+0000 front 2024-04-25T16:55:21.556247+0000 (oldest deadline 2024-04-25T16:55:43.256356+0000) 2024-04-25T16:55:44.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:43.549+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:44.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:43.664+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:44.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:43.664+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:44.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:43 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:43.664+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:44.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:43.814+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:44.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:43 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:43.814+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:44.698 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:44 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:44.226+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:44.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:44 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:44.263+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:44.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:44 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:44.263+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:44.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:44 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:44.451+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:44.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:44 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:44.451+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:44.699 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:44 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:44.451+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.556207+0000 front 2024-04-25T16:55:21.556247+0000 (oldest deadline 2024-04-25T16:55:43.256356+0000) 2024-04-25T16:55:44.771 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:44.489+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:44.771 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:44.489+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:45.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:44 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:44.563+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:45.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:44 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:44.700+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:45.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:44 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:44.700+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:45.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:44 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:44.700+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:45.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:44.773+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:45.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:44.773+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:45.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:44 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:44.773+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:45.477 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:45.217+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:45.477 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:45.218+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:45.477 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:45.250+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:45.478 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:45.250+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:45.740 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:45.487+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:45.740 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:45.487+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:45.740 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:45.487+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:45.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:45.480+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:02.253147+0000 front 2024-04-25T16:55:02.253224+0000 (oldest deadline 2024-04-25T16:55:25.753094+0000) 2024-04-25T16:55:45.949 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:45.480+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.555066+0000 front 2024-04-25T16:55:11.555159+0000 (oldest deadline 2024-04-25T16:55:35.655041+0000) 2024-04-25T16:55:45.949 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:45 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5[44804]: 2024-04-25T16:55:45.480+0000 7f23b14ae700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.556207+0000 front 2024-04-25T16:55:21.556247+0000 (oldest deadline 2024-04-25T16:55:43.256356+0000) 2024-04-25T16:55:46.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:45 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:45.590+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:46.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:45 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:45.590+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:46.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:45 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:45.727+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:46.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:45 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:45.727+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:46.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:45 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:45.727+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:46.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:45.742+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:46.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:45.742+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:46.096 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:45 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:45.742+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:46.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:46 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:46.239+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:46.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:46 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:46.239+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:46.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:46 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:46.174+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:46.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:46 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:46.174+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:46.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:46.696+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:46.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:46.696+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:46.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:46.696+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:46.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:46.479+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:46.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:46.479+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:46.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:46.479+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:47.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:46 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:46.566+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:47.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:46 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:46.566+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:47.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:46 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:46.684+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:47.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:46 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:46.684+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:47.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:46 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:46.684+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:47.313 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:47 smithi133 podman[70774]: 2024-04-25 16:55:47.05249683 +0000 UTC m=+5.737209996 container died 0aa3eebda048858d99c5c089b4f7324b00fb6c26b9ae857f04db8f986e2b604c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T16:55:47.314 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:47 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:47.219+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:47.314 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:47 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:47.219+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:47.314 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:47 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:47.219+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:47.314 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:47 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:47.225+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:47.314 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:47 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:47.225+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:47.697 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:47 smithi133 podman[70774]: 2024-04-25 16:55:47.407404707 +0000 UTC m=+6.092117872 container cleanup 0aa3eebda048858d99c5c089b4f7324b00fb6c26b9ae857f04db8f986e2b604c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:55:47.697 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:47 smithi133 bash[70774]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5 2024-04-25T16:55:47.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:47.680+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:47.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:47.680+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:47.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:47.680+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:47.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:47.523+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:47.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:47.523+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:47.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:47 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:47.523+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:47.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:47 smithi133 podman[70791]: 2024-04-25 16:55:47.700461918 +0000 UTC m=+0.646538758 container remove 0aa3eebda048858d99c5c089b4f7324b00fb6c26b9ae857f04db8f986e2b604c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T16:55:48.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:47.520+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:48.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:47.520+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:48.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:47.682+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:48.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:47.682+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:48.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:47 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:47.682+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:48.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:48 smithi133 podman[70883]: 2024-04-25 16:55:47.922928027 +0000 UTC m=+0.018179415 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:48.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:48 smithi133 podman[70883]: 2024-04-25 16:55:48.051629187 +0000 UTC m=+0.146880561 container create cf48bffc1d2bdf17e37a97e05693288d27fa85f4e255bdeca0eb19c686c4fdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-25T16:55:48.301 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:48 smithi133 podman[70883]: 2024-04-25 16:55:48.304289649 +0000 UTC m=+0.399541023 container init cf48bffc1d2bdf17e37a97e05693288d27fa85f4e255bdeca0eb19c686c4fdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:55:48.302 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:48.259+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:48.302 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:48.259+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:48.302 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:48.176+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:48.302 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:48.176+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:48.302 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:48 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:48.176+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:48.595 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:48 smithi133 podman[70883]: 2024-04-25 16:55:48.307661286 +0000 UTC m=+0.402912661 container start cf48bffc1d2bdf17e37a97e05693288d27fa85f4e255bdeca0eb19c686c4fdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T16:55:48.596 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:48 smithi133 podman[70883]: 2024-04-25 16:55:48.387479731 +0000 UTC m=+0.482731108 container attach cf48bffc1d2bdf17e37a97e05693288d27fa85f4e255bdeca0eb19c686c4fdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T16:55:48.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:48.544+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:48.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:48.544+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:48.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:48.544+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:48.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:48.673+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:48.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:48.673+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:48.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:48 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:48.673+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:48.948 INFO:journalctl@ceph.osd.5.smithi133.stdout:Apr 25 16:55:48 smithi133 podman[70883]: 2024-04-25 16:55:48.599191583 +0000 UTC m=+0.694442967 container died cf48bffc1d2bdf17e37a97e05693288d27fa85f4e255bdeca0eb19c686c4fdbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:55:49.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:48.527+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:49.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:48.527+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:49.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:48.638+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:49.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:48.638+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:49.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:48 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:48.638+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:49.280 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:49 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:49.206+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:49.281 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:49 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:49.206+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:49.281 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:49 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:49.206+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:49.307 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T16:55:49.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:49.346 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T16:55:49.346 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T16:55:49.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.6 2024-04-25T16:55:49.630 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 systemd[1]: Stopping Ceph osd.6 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:49.631 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:49.497+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:49.631 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:49.497+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:49.631 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:49.497+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:49.698 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:49 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:49.282+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:49.699 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:49 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:49.282+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:50.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:49 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:49.540+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:50.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:49 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:49.540+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:50.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:49 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:49.649+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:50.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:49 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:49.649+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:50.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:49 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:49.649+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:50.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:49.633+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:50.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:49.633+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:50.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:49.633+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:50.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:49.676+0000 7f6ba6033700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:50.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:49.676+0000 7f6ba6033700 -1 osd.6 112 *** Got signal Terminated *** 2024-04-25T16:55:50.095 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:49 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:49.676+0000 7f6ba6033700 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:50.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:50 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:50.246+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:50.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:50 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:50.246+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:50.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:50 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:50.161+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:50.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:50 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:50.161+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:50.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:50 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:50.161+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:50.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:50 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:50.610+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:50.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:50 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:50.610+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:50.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:50 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:50.610+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:50.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:50 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:50.537+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:50.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:50 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:50.537+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:50.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:50 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:50.537+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:51.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:50 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:50.558+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:51.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:50 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:50.558+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:51.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:50 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:50.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:51.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:50 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:50.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:51.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:50 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:50.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:51.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:51 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:51.234+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:51.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:51 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:51.234+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:51.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:51 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:51.171+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:51.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:51 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:51.171+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:51.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:51 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:51.171+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:51.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:51.576+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:51.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:51.576+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:51.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:51.576+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:51.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:51.525+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:51.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:51.525+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:51.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:51.525+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:51.846 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:51.525+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:52.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:51 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:51.585+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:52.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:51 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:51.585+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:52.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:51 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:51.581+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:52.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:51 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:51.581+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:52.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:51 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:51.581+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:52.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:52 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:52.141+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:52.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:52 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:52.141+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:52.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:52 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:52.141+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:52.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:52 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:52.196+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:52.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:52 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:52.196+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:52.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:52 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:52.196+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:52.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:52.622+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:52.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:52.622+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:52.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:52.622+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:52.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:52.622+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:30.002210+0000 front 2024-04-25T16:55:30.002232+0000 (oldest deadline 2024-04-25T16:55:51.702336+0000) 2024-04-25T16:55:52.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:52.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:52.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:52.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:52.846 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:52.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:52.846 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:52 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:52.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:53.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:52.612+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:53.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:52.612+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:53.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:52.612+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:53.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:52.616+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:53.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:52.616+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:53.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:52.616+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:53.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:52 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:52.616+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:53.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:53 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:53.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:53.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:53 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:53.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:53.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:53 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:53.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:53.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:53 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:53.151+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:53.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:53 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:53.151+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:53.450 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:53 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:53.151+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:53.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:53.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:53.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:53.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:53.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:53.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:53.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:53.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:53.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:53.630+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:53.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:53.630+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:53.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:53.630+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:53.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:53 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:53.630+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:30.002210+0000 front 2024-04-25T16:55:30.002232+0000 (oldest deadline 2024-04-25T16:55:51.702336+0000) 2024-04-25T16:55:54.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:53.631+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:54.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:53.631+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:54.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:53.631+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:54.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:53.657+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:54.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:53.657+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:54.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:53.657+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:54.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:53 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:53.657+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:54.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:54 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:54.193+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:54.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:54 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:54.193+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:54.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:54 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:54.193+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:54.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:54 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:54.109+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:54.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:54 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:54.109+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:54.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:54 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:54.109+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:54.844 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:54.607+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:04.798034+0000 front 2024-04-25T16:55:04.798068+0000 (oldest deadline 2024-04-25T16:55:27.698217+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:54.607+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.599808+0000 front 2024-04-25T16:55:13.599696+0000 (oldest deadline 2024-04-25T16:55:38.899658+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:54.607+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.600899+0000 front 2024-04-25T16:55:20.600842+0000 (oldest deadline 2024-04-25T16:55:44.100953+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6[58168]: 2024-04-25T16:55:54.607+0000 7f6b9e5e1700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:30.002210+0000 front 2024-04-25T16:55:30.002232+0000 (oldest deadline 2024-04-25T16:55:51.702336+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:54.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:54.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:54.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:54.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:54 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:54.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:55.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:54.679+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:55.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:54.679+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:55.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:54.679+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:55.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:54.697+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:55.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:54.697+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:55.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:54.697+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:55.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:54 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:54.697+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:55.401 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:55 smithi112 podman[92605]: 2024-04-25 16:55:55.141199899 +0000 UTC m=+5.589111541 container died 46b50f21cd5d7fbbb366aff0c72d7733c3d0a0817113fddd2b49267244df948d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:55:55.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:55.173+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:55.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:55.173+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:55.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:55.173+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:55.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:55.089+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:55.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:55.089+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:55.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:55 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:55.089+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:55.768 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:55 smithi112 podman[92605]: 2024-04-25 16:55:55.495442051 +0000 UTC m=+5.943353700 container cleanup 46b50f21cd5d7fbbb366aff0c72d7733c3d0a0817113fddd2b49267244df948d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T16:55:55.769 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:55 smithi112 bash[92605]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6 2024-04-25T16:55:55.769 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:55 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:55.507+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:55.769 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:55 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:55.507+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:55.769 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:55 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:55.507+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:55.769 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:55 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:55.507+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:56.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:55.665+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:56.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:55.665+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:56.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:55.665+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:56.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:55.690+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:56.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:55.690+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:56.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:55.690+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:56.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:55.690+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:56.094 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:55 smithi112 podman[92622]: 2024-04-25 16:55:55.771756601 +0000 UTC m=+0.631693881 container remove 46b50f21cd5d7fbbb366aff0c72d7733c3d0a0817113fddd2b49267244df948d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T16:55:56.372 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:56 smithi112 podman[92712]: 2024-04-25 16:55:56.020069681 +0000 UTC m=+0.018159257 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:55:56.372 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:56 smithi112 podman[92712]: 2024-04-25 16:55:56.139378012 +0000 UTC m=+0.137467578 container create b62ff2b1e3eeb295ced812e4a20711176858265e42acd3deaa44800ecf1f6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T16:55:56.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:56.139+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:56.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:56.139+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:56.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:56.139+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:56.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:56.090+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:56.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:56.090+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:56.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:56 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:56.090+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:56.664 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:56 smithi112 podman[92712]: 2024-04-25 16:55:56.375360775 +0000 UTC m=+0.373450341 container init b62ff2b1e3eeb295ced812e4a20711176858265e42acd3deaa44800ecf1f6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:55:56.664 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:56 smithi112 podman[92712]: 2024-04-25 16:55:56.378754724 +0000 UTC m=+0.376844294 container start b62ff2b1e3eeb295ced812e4a20711176858265e42acd3deaa44800ecf1f6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T16:55:56.664 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:56 smithi112 podman[92712]: 2024-04-25 16:55:56.492123481 +0000 UTC m=+0.490213049 container attach b62ff2b1e3eeb295ced812e4a20711176858265e42acd3deaa44800ecf1f6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-25T16:55:56.664 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:56 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:56.532+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:56.664 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:56 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:56.532+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:56.664 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:56 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:56.532+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:56.665 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:56 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:56.532+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:56.975 INFO:journalctl@ceph.osd.6.smithi112.stdout:Apr 25 16:55:56 smithi112 podman[92712]: 2024-04-25 16:55:56.66722581 +0000 UTC m=+0.665315380 container died b62ff2b1e3eeb295ced812e4a20711176858265e42acd3deaa44800ecf1f6395 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T16:55:57.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:56.651+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:57.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:56.651+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:57.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:56.651+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:57.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:56.685+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:57.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:56.685+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:57.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:56.685+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:57.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:56 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:56.685+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:57.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T16:55:57.343 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:55:57.343 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T16:55:57.344 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T16:55:57.344 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.7 2024-04-25T16:55:57.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:57 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:57.149+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:57.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:57 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:57.149+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:57.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:57 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:57.149+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:57.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:57 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:57.083+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:57.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:57 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:57.083+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:57.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:57 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:57.083+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:57.555 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:57 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:57.506+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:57.555 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:57 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:57.506+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:57.555 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:57 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:57.506+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:57.555 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:57 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:57.506+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:57.624 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 systemd[1]: Stopping Ceph osd.7 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:55:58.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:57.647+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:58.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:57.647+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:58.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:57.647+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:57.647+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:57.627+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:57.627+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:57.627+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:57.655+0000 7efc95084700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:57.655+0000 7efc95084700 -1 osd.7 112 *** Got signal Terminated *** 2024-04-25T16:55:58.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:57 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:57.655+0000 7efc95084700 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:55:58.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:58 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:58.138+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:58.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:58 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:58.138+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:58.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:58 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:58.138+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:58.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:58 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:58.067+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:58.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:58 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:58.067+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:58.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:58 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:58.067+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:58.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:58 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:58.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:58.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:58 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:58.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:58.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:58 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:58.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:58.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:58 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:58.464+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:55:59.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:58.625+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:55:59.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:58.625+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:55:59.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:58.625+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:55:59.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:58.658+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:55:59.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:58.658+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:55:59.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:58.658+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:55:59.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:58.658+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:55:59.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:59 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:59.097+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:55:59.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:59 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:59.097+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:55:59.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:55:59 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:55:59.097+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:55:59.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:59 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:59.092+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:55:59.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:59 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:59.092+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:55:59.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:55:59 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:55:59.092+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:55:59.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:59 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:59.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:55:59.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:59 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:59.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:55:59.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:59 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:59.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:55:59.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:55:59 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:55:59.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:00.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:59.579+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:56:00.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:59.579+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:56:00.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:59.579+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:56:00.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:55:59.579+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.213595+0000 front 2024-04-25T16:55:35.213549+0000 (oldest deadline 2024-04-25T16:55:59.313650+0000) 2024-04-25T16:56:00.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:59.623+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:00.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:59.623+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:00.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:59.623+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:00.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:55:59 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:55:59.623+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:00.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:00 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:00.060+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:00.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:00 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:00.060+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:00.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:00 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:00.060+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:00.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:00 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:00.071+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:00.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:00 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:00.071+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:00.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:00 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:00.071+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:00.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:00.447+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:00.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:00.447+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:00.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:00.447+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:00.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:00 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:00.447+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:01.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:00.653+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:01.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:00.653+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:01.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:00.653+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:01.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:00.653+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:01.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:00.623+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:56:01.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:00.623+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:56:01.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:00.623+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:56:01.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:00.623+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.213595+0000 front 2024-04-25T16:55:35.213549+0000 (oldest deadline 2024-04-25T16:55:59.313650+0000) 2024-04-25T16:56:01.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:01 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:01.034+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:01.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:01 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:01.034+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:01.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:01 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:01.034+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:01.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:01 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:01.092+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:01.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:01 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:01.092+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:01.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:01 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:01.092+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:01.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:01 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:01.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:01.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:01 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:01.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:01.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:01 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:01.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:01.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:01 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:01.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:02.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:01.602+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:56:02.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:01.602+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:56:02.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:01.602+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:56:02.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:01.602+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.213595+0000 front 2024-04-25T16:55:35.213549+0000 (oldest deadline 2024-04-25T16:55:59.313650+0000) 2024-04-25T16:56:02.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:01.632+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:02.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:01.632+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:02.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:01.632+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:02.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:01 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:01.632+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:02.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:02.089+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:02.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:02.089+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:02.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:02.089+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:02.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.045+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:02.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.045+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:02.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.045+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:02.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.045+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:02.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:02 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:02.505+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:02.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:02 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:02.505+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:02.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:02 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:02.505+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:02.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:02 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:02.505+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:03.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:02.561+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.309631+0000 front 2024-04-25T16:55:11.309662+0000 (oldest deadline 2024-04-25T16:55:37.209792+0000) 2024-04-25T16:56:03.007 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:02.561+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:20.111163+0000 front 2024-04-25T16:55:20.110952+0000 (oldest deadline 2024-04-25T16:55:44.810919+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:02.561+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.212733+0000 front 2024-04-25T16:55:28.212818+0000 (oldest deadline 2024-04-25T16:55:51.712601+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7[50967]: 2024-04-25T16:56:02.561+0000 7efc9163a700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.213595+0000 front 2024-04-25T16:55:35.213549+0000 (oldest deadline 2024-04-25T16:55:59.313650+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:02.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:02.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:02.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:02.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:03.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:02 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:02.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:03.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.997+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:03.450 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.997+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:03.450 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.997+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:03.450 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:02 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:02.997+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:03.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:03.078+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:03.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:03.078+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:03.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:03.078+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:03.451 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:03.078+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:03.507 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:03 smithi116 podman[71203]: 2024-04-25 16:56:03.234624473 +0000 UTC m=+5.697763657 container died 727f721aa43d35b4d61b0a1e4750dab1f2130058fdd6ac32ae3f05a17e4d09aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:56:03.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:03 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:03.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:03.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:03 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:03.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:03.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:03 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:03.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:03.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:03 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:03.470+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:03.938 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:03 smithi116 podman[71203]: 2024-04-25 16:56:03.631326465 +0000 UTC m=+6.094465639 container cleanup 727f721aa43d35b4d61b0a1e4750dab1f2130058fdd6ac32ae3f05a17e4d09aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T16:56:03.938 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:03 smithi116 bash[71203]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7 2024-04-25T16:56:03.938 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:03 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:03.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:03.939 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:03 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:03.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:03.939 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:03 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:03.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:03.939 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:03 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:03.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:03.939 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:03 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:03.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:04.247 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:03 smithi116 podman[71220]: 2024-04-25 16:56:03.941320015 +0000 UTC m=+0.707338246 container remove 727f721aa43d35b4d61b0a1e4750dab1f2130058fdd6ac32ae3f05a17e4d09aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T16:56:04.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:03.984+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:04.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:03.984+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:04.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:03.984+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:04.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:03 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:03.984+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:04.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:04.066+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:04.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:04.066+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:04.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:04.066+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:04.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:04.066+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:04.500 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:04 smithi116 podman[71311]: 2024-04-25 16:56:04.250401704 +0000 UTC m=+0.110819624 container create b322714645ff23634b3d7a0f5164f0ab9eeb026cde87caaa823dd816588b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T16:56:04.501 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:04 smithi116 podman[71311]: 2024-04-25 16:56:04.15746977 +0000 UTC m=+0.017887707 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:56:04.501 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:04 smithi116 podman[71311]: 2024-04-25 16:56:04.503478776 +0000 UTC m=+0.363896709 container init b322714645ff23634b3d7a0f5164f0ab9eeb026cde87caaa823dd816588b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T16:56:04.757 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:04 smithi116 podman[71311]: 2024-04-25 16:56:04.5070196 +0000 UTC m=+0.367437522 container start b322714645ff23634b3d7a0f5164f0ab9eeb026cde87caaa823dd816588b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T16:56:04.758 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:04 smithi116 podman[71311]: 2024-04-25 16:56:04.586541249 +0000 UTC m=+0.446959184 container attach b322714645ff23634b3d7a0f5164f0ab9eeb026cde87caaa823dd816588b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T16:56:04.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:04 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:04.637+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:04.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:04 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:04.637+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:04.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:04 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:04.637+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:04.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:04 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:04.637+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:04.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:04 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:04.637+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:04.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:04 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:04.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:04.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:04 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:04.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:04.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:04 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:04.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:04.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:04 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:04.430+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:05.136 INFO:journalctl@ceph.osd.7.smithi116.stdout:Apr 25 16:56:04 smithi116 podman[71311]: 2024-04-25 16:56:04.807777364 +0000 UTC m=+0.668195285 container died b322714645ff23634b3d7a0f5164f0ab9eeb026cde87caaa823dd816588b8025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:56:05.448 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:04.977+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:05.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:04.977+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:05.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:04.977+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:05.449 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:04 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:04.977+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:05.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:05.107+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:05.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:05.107+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:05.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:05.107+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:05.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:05.107+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:05.463 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T16:56:05.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:05.501 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T16:56:05.502 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T16:56:05.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.8 2024-04-25T16:56:05.715 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:05.596+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:05.715 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:05.596+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:05.715 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:05.596+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:05.715 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:05.596+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:05.716 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:05.596+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:05.834 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 systemd[1]: Stopping Ceph osd.8 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:05.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:05 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:05.476+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:05.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:05 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:05.476+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:05.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:05 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:05.476+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:05.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:05 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:05.476+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:06.126 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.836+0000 7f40055ae700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:06.127 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.836+0000 7f40055ae700 -1 osd.8 112 *** Got signal Terminated *** 2024-04-25T16:56:06.127 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.836+0000 7f40055ae700 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:56:06.127 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.951+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:06.127 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.951+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:06.127 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.951+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:06.127 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:05 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:05.951+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:06.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:06.128+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:06.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:06.128+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:06.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:06.128+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:06.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:06.128+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:06.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:06 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:06.504+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:06.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:06 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:06.504+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:06.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:06 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:06.504+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:06.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:06 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:06.504+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:07.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:06 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:06.548+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:07.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:06 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:06.548+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:07.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:06 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:06.548+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:07.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:06 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:06.548+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:07.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:06 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:06.548+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:07.198 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:06.925+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:07.199 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:06.925+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:07.199 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:06.925+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:07.199 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:06 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:06.925+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:07.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:07.173+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:07.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:07.173+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:07.200 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:07.173+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:07.200 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:07.173+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:07.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:07 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:07.483+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:07.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:07 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:07.483+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:07.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:07 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:07.483+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:07.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:07 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:07.483+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:07.908 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:07 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:07.534+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:07.909 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:07 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:07.534+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:07.909 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:07 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:07.534+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:07.909 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:07 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:07.534+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:07.909 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:07 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:07.534+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:08.386 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:08.125+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:08.386 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:08.125+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:08.386 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:08.125+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:08.387 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:08.125+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:08.387 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:07.967+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:08.387 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:07.967+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:08.387 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:07.967+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:08.387 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:07 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:07.967+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:08.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:08.437+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:08.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:08.437+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:08.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:08.437+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:08.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:08 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:08.437+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:09.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:08.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:09.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:08.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:09.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:08.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:09.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:08.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:09.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:08 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:08.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:09.198 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:09.099+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:09.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:09.099+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:09.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:09.099+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:09.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:09.099+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:09.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:09.099+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:09.199 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:08.923+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:09.199 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:08.923+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:09.200 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:08.923+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:09.200 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:08.923+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:09.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:09 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:09.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:09.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:09 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:09.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:09.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:09 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:09.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:09.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:09 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:09.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:10.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:09.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:10.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:09.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:10.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:09.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:10.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:09.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:10.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:09 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:09.609+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:10.131 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:09.882+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.137855+0000 front 2024-04-25T16:55:13.137836+0000 (oldest deadline 2024-04-25T16:55:38.338444+0000) 2024-04-25T16:56:10.131 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:09.882+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:18.338980+0000 front 2024-04-25T16:55:18.339018+0000 (oldest deadline 2024-04-25T16:55:44.239130+0000) 2024-04-25T16:56:10.131 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:09.882+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:28.940102+0000 front 2024-04-25T16:55:28.940385+0000 (oldest deadline 2024-04-25T16:55:51.740545+0000) 2024-04-25T16:56:10.132 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:09.882+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:35.241697+0000 front 2024-04-25T16:55:35.241921+0000 (oldest deadline 2024-04-25T16:56:01.141506+0000) 2024-04-25T16:56:10.132 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:09 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8[51555]: 2024-04-25T16:56:09.882+0000 7f4001b64700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.242471+0000 front 2024-04-25T16:55:45.242443+0000 (oldest deadline 2024-04-25T16:56:09.342556+0000) 2024-04-25T16:56:10.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:10 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:10.134+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:10.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:10 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:10.134+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:10.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:10 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:10.134+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:10.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:10 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:10.134+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:10.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:10 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:10.134+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:10.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:10.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:10.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:10.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:10.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:10.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:10.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:10 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:10.463+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:11.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:10 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:10.592+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:11.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:10 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:10.592+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:11.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:10 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:10.592+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:11.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:10 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:10.592+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:11.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:10 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:10.592+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:11.361 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:11 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:11.085+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:11.361 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:11 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:11.085+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:11.362 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:11 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:11.085+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:11.362 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:11 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:11.085+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:11.362 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:11 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:11.085+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:11.662 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:11 smithi133 podman[70990]: 2024-04-25 16:56:11.364210215 +0000 UTC m=+5.659174942 container died 88e4787846cb064b1f9e135f023563e9621926b1801d0e7e7cf28067bf0fff7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T16:56:11.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:11.496+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:11.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:11.496+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:11.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:11.496+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:11.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:11.496+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:11.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:11 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:11.496+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:11.948 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:11 smithi133 podman[70990]: 2024-04-25 16:56:11.770981409 +0000 UTC m=+6.065946125 container cleanup 88e4787846cb064b1f9e135f023563e9621926b1801d0e7e7cf28067bf0fff7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T16:56:11.949 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:11 smithi133 bash[70990]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8 2024-04-25T16:56:12.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:11 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:11.620+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:12.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:11 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:11.620+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:12.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:11 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:11.620+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:12.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:11 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:11.620+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:12.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:11 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:11.620+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:12.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:11 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:11.620+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:12.325 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71020]: 2024-04-25 16:56:12.008222347 +0000 UTC m=+0.642083543 container remove 88e4787846cb064b1f9e135f023563e9621926b1801d0e7e7cf28067bf0fff7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-25T16:56:12.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:12 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:12.102+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:12.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:12 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:12.102+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:12.325 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:12 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:12.102+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:12.326 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:12 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:12.102+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:12.326 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:12 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:12.102+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:12.600 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71115]: 2024-04-25 16:56:12.231443263 +0000 UTC m=+0.017931856 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:56:12.600 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71115]: 2024-04-25 16:56:12.375708445 +0000 UTC m=+0.162197026 container create 0e1ff39d593f34729e057b805bbb07b7f532414868a0ed41662e7376b2be4b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T16:56:12.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:12.468+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:12.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:12.468+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:12.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:12.468+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:12.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:12.468+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:12.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:12 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:12.468+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:12.893 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71115]: 2024-04-25 16:56:12.603818338 +0000 UTC m=+0.390306919 container init 0e1ff39d593f34729e057b805bbb07b7f532414868a0ed41662e7376b2be4b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T16:56:12.894 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71115]: 2024-04-25 16:56:12.607032434 +0000 UTC m=+0.393521026 container start 0e1ff39d593f34729e057b805bbb07b7f532414868a0ed41662e7376b2be4b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T16:56:12.894 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71115]: 2024-04-25 16:56:12.720678103 +0000 UTC m=+0.507166696 container attach 0e1ff39d593f34729e057b805bbb07b7f532414868a0ed41662e7376b2be4b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T16:56:13.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:12 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:12.597+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:13.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:12 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:12.597+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:13.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:12 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:12.597+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:13.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:12 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:12.597+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:13.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:12 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:12.597+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:13.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:12 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:12.597+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:13.199 INFO:journalctl@ceph.osd.8.smithi133.stdout:Apr 25 16:56:12 smithi133 podman[71115]: 2024-04-25 16:56:12.897356802 +0000 UTC m=+0.683845385 container died 0e1ff39d593f34729e057b805bbb07b7f532414868a0ed41662e7376b2be4b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-8-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:56:13.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:13.131+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:13.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:13.131+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:13.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:13.131+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:13.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:13.131+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:13.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:13.131+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:13.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T16:56:13.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:13.613 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T16:56:13.614 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T16:56:13.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.9 2024-04-25T16:56:13.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.458+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:13.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.458+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:13.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.458+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:13.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.458+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:13.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.458+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:13.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 systemd[1]: Stopping Ceph osd.9 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:14.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:13 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:13.585+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:14.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:13 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:13.585+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:14.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:13 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:13.585+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:14.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:13 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:13.585+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:14.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:13 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:13.585+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:14.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:13 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:13.585+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:14.344 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.940+0000 7f99cca45700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:14.345 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.940+0000 7f99cca45700 -1 osd.9 112 *** Got signal Terminated *** 2024-04-25T16:56:14.345 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:13 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:13.940+0000 7f99cca45700 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:56:14.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:14 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:14.130+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:14.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:14 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:14.130+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:14.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:14 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:14.130+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:14.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:14 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:14.130+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:14.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:14 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:14.130+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:14.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:14.494+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:14.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:14.494+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:14.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:14.494+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:14.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:14.494+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:14.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:14 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:14.494+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:15.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:14 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:14.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:15.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:14 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:14.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:15.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:14 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:14.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:15.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:14 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:14.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:15.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:14 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:14.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:15.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:14 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:14.615+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:15.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:15 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:15.165+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:15.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:15 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:15.165+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:15.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:15 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:15.165+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:15.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:15 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:15.165+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:15.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:15 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:15.165+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:15.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:15.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:15.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:15.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:15.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:15.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:15.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:15.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:15.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:15 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:15.465+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:16.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:15.636+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:16.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:15.636+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:16.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:15.636+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:16.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:15.636+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:16.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:15.636+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:16.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:15 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:15.636+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:16.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:16 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:16.123+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:16.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:16 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:16.123+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:16.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:16 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:16.123+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:16.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:16 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:16.123+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:16.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:16 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:16.123+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:16.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:16 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:16.123+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:16.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:16 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:16.514+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:16.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:16 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:16.514+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:16.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:16 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:16.514+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:16.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:16 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:16.514+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:16.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:16 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:16.514+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:17.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:17.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:17.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:17.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:17.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:17.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:17.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:16 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:16.621+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:17.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:17.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:17.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:17.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:17.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:17.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:17.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:17.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:17.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:17.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:17.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:17 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:17.135+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:17.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:17.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:17.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:17.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:17.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:17.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:17.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:17.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:17.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:17.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:17.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:17 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:17.491+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.244418+0000 front 2024-04-25T16:55:51.244488+0000 (oldest deadline 2024-04-25T16:56:17.144619+0000) 2024-04-25T16:56:17.963 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:17.964 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:17.964 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:17.964 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:17.964 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:17.964 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:17.964 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:17 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:17.633+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:18.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:18.178+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:18.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:18.178+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:18.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:18.178+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:18.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:18.178+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:18.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:18.178+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:18.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:18.178+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:18.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:18.524+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.035771+0000 front 2024-04-25T16:55:01.035850+0000 (oldest deadline 2024-04-25T16:55:26.935833+0000) 2024-04-25T16:56:18.844 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:18.524+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:13.337405+0000 front 2024-04-25T16:55:13.337315+0000 (oldest deadline 2024-04-25T16:55:38.037496+0000) 2024-04-25T16:56:18.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:18.524+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:19.738907+0000 front 2024-04-25T16:55:19.738888+0000 (oldest deadline 2024-04-25T16:55:45.039018+0000) 2024-04-25T16:56:18.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:18.524+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:29.140256+0000 front 2024-04-25T16:55:29.140433+0000 (oldest deadline 2024-04-25T16:55:50.840461+0000) 2024-04-25T16:56:18.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:18.524+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.943999+0000 front 2024-04-25T16:55:45.944162+0000 (oldest deadline 2024-04-25T16:56:11.243890+0000) 2024-04-25T16:56:18.845 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:18 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9[65065]: 2024-04-25T16:56:18.524+0000 7f99c4ff3700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.244418+0000 front 2024-04-25T16:55:51.244488+0000 (oldest deadline 2024-04-25T16:56:17.144619+0000) 2024-04-25T16:56:19.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:19.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:19.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:19.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:19.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:19.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:19.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:18 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:18.608+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:19.688 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:19 smithi112 podman[92832]: 2024-04-25 16:56:19.374334007 +0000 UTC m=+5.559470504 container died bd387d1427a0233034639fc39adb5580db356aad6d0d49407b9442456429fd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T16:56:19.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:19 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:19.202+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:19.698 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:19 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:19.202+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:19.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:19 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:19.202+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:19.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:19 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:19.202+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:19.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:19 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:19.202+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:19.699 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:19 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:19.202+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:19.996 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:19 smithi112 podman[92832]: 2024-04-25 16:56:19.790446842 +0000 UTC m=+5.975583341 container cleanup bd387d1427a0233034639fc39adb5580db356aad6d0d49407b9442456429fd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T16:56:19.996 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:19 smithi112 bash[92832]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9 2024-04-25T16:56:20.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:20.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:20.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:20.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:20.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:20.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:20.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:19 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:19.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:20.344 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92849]: 2024-04-25 16:56:19.999888768 +0000 UTC m=+0.625721965 container remove bd387d1427a0233034639fc39adb5580db356aad6d0d49407b9442456429fd28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:56:20.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:20.156+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:20.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:20.156+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:20.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:20.156+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:20.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:20.156+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:20.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:20.156+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:20.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:20 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:20.156+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:20.666 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92939]: 2024-04-25 16:56:20.311411978 +0000 UTC m=+0.017726450 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:56:20.667 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92939]: 2024-04-25 16:56:20.442702534 +0000 UTC m=+0.149017013 container create 98ec81d17b60e4628671b13cc21fddae25070668f7cd5cb4082649be5f5c9d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T16:56:20.667 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92939]: 2024-04-25 16:56:20.669958129 +0000 UTC m=+0.376272566 container init 98ec81d17b60e4628671b13cc21fddae25070668f7cd5cb4082649be5f5c9d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:56:20.962 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92939]: 2024-04-25 16:56:20.673569408 +0000 UTC m=+0.379883850 container start 98ec81d17b60e4628671b13cc21fddae25070668f7cd5cb4082649be5f5c9d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T16:56:20.962 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92939]: 2024-04-25 16:56:20.778469881 +0000 UTC m=+0.484784319 container attach 98ec81d17b60e4628671b13cc21fddae25070668f7cd5cb4082649be5f5c9d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T16:56:21.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:21.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:21.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:21.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:21.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:21.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:21.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:20 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:20.589+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:21.344 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 conmon[92954]: conmon 98ec81d17b60e4628671 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-98ec81d17b60e4628671b13cc21fddae25070668f7cd5cb4082649be5f5c9d40.scope/container/memory.events 2024-04-25T16:56:21.344 INFO:journalctl@ceph.osd.9.smithi112.stdout:Apr 25 16:56:20 smithi112 podman[92939]: 2024-04-25 16:56:20.966548463 +0000 UTC m=+0.672862917 container died 98ec81d17b60e4628671b13cc21fddae25070668f7cd5cb4082649be5f5c9d40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T16:56:21.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T16:56:21.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:21.115+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:21.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:21.115+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:21.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:21.115+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:21.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:21.115+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:21.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:21.115+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:21.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:21 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:21.115+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:21.456 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:21.457 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T16:56:21.457 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T16:56:21.457 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.10 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 systemd[1]: Stopping Ceph osd.10 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:21.758 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:21.759 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.622+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:22.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.809+0000 7f17fd6ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:22.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.809+0000 7f17fd6ea700 -1 osd.10 112 *** Got signal Terminated *** 2024-04-25T16:56:22.258 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:21 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:21.809+0000 7f17fd6ea700 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:56:22.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:22.084+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:22.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:22.084+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:22.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:22.084+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:22.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:22.084+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:22.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:22.084+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:22.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:22.084+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:23.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:23.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:23.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:23.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:23.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:23.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:23.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:22 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:22.601+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:23.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:23.087+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:23.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:23.087+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:23.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:23.087+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:23.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:23.087+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:23.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:23.087+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:23.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:23 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:23.087+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:24.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:24.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:24.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:24.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:24.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:24.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:24.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:23 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:23.582+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:24.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:24.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:24.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:24.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:24.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:24.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:24.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:24.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:24.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:24.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:24.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:24 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:24.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:25.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:25.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:25.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:25.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:25.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:25.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:25.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:24 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:24.603+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:25.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:25.075+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:25.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:25.075+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:25.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:25.075+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:25.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:25.075+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:25.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:25.075+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:25.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:25 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:25.075+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:26.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:26.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:26.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:26.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:26.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:26.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:26.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:25 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:25.613+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:26.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:26.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:26.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:26.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:26.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:26.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:26.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:26 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:26.119+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:27.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:03.400005+0000 front 2024-04-25T16:55:03.400065+0000 (oldest deadline 2024-04-25T16:55:26.300086+0000) 2024-04-25T16:56:27.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:09.701835+0000 front 2024-04-25T16:55:09.701868+0000 (oldest deadline 2024-04-25T16:55:34.402150+0000) 2024-04-25T16:56:27.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:21.304488+0000 front 2024-04-25T16:55:21.304385+0000 (oldest deadline 2024-04-25T16:55:43.604062+0000) 2024-04-25T16:56:27.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6814 osd.3 since back 2024-04-25T16:55:27.105274+0000 front 2024-04-25T16:55:27.105342+0000 (oldest deadline 2024-04-25T16:55:51.805381+0000) 2024-04-25T16:56:27.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.607080+0000 front 2024-04-25T16:55:37.607105+0000 (oldest deadline 2024-04-25T16:56:02.307204+0000) 2024-04-25T16:56:27.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:45.808267+0000 front 2024-04-25T16:55:45.808376+0000 (oldest deadline 2024-04-25T16:56:11.608893+0000) 2024-04-25T16:56:27.008 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:26 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10[57876]: 2024-04-25T16:56:26.641+0000 7f17f9ca0700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:51.609567+0000 front 2024-04-25T16:55:51.609612+0000 (oldest deadline 2024-04-25T16:56:16.309512+0000) 2024-04-25T16:56:27.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:27.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:27.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:27.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.076+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:27.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.077+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:27.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.077+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:27.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:27 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:27.077+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:27.591 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:27 smithi116 podman[71428]: 2024-04-25 16:56:27.331459994 +0000 UTC m=+5.666166552 container died 2e88cce8072cfebb5ba85afff4e91f1fa65492d30c0bc59bff25584bcf5ed18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T16:56:27.956 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:27 smithi116 podman[71428]: 2024-04-25 16:56:27.684010797 +0000 UTC m=+6.018717361 container cleanup 2e88cce8072cfebb5ba85afff4e91f1fa65492d30c0bc59bff25584bcf5ed18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T16:56:27.957 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:27 smithi116 bash[71428]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10 2024-04-25T16:56:28.257 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:27 smithi116 podman[71445]: 2024-04-25 16:56:27.960109393 +0000 UTC m=+0.625489085 container remove 2e88cce8072cfebb5ba85afff4e91f1fa65492d30c0bc59bff25584bcf5ed18f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:28.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:28 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:28.058+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:28.593 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:28 smithi116 podman[71539]: 2024-04-25 16:56:28.202030399 +0000 UTC m=+0.017763052 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:56:28.593 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:28 smithi116 podman[71539]: 2024-04-25 16:56:28.302548659 +0000 UTC m=+0.118281287 container create 1473aef65529e2b2f401c473b058ae845e5c1341f4d3f0e0c1aca9476d205ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T16:56:28.594 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:28 smithi116 podman[71539]: 2024-04-25 16:56:28.513771014 +0000 UTC m=+0.329503655 container init 1473aef65529e2b2f401c473b058ae845e5c1341f4d3f0e0c1aca9476d205ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:56:28.594 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:28 smithi116 podman[71539]: 2024-04-25 16:56:28.51714949 +0000 UTC m=+0.332882118 container start 1473aef65529e2b2f401c473b058ae845e5c1341f4d3f0e0c1aca9476d205ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:56:28.594 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:28 smithi116 podman[71539]: 2024-04-25 16:56:28.596587298 +0000 UTC m=+0.412319935 container attach 1473aef65529e2b2f401c473b058ae845e5c1341f4d3f0e0c1aca9476d205ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T16:56:29.007 INFO:journalctl@ceph.osd.10.smithi116.stdout:Apr 25 16:56:28 smithi116 podman[71539]: 2024-04-25 16:56:28.810359332 +0000 UTC m=+0.626091969 container died 1473aef65529e2b2f401c473b058ae845e5c1341f4d3f0e0c1aca9476d205ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T16:56:29.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:29.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:29.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:29.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:29.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:29.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:29.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.047+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:29.483 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T16:56:29.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:29.520 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T16:56:29.520 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T16:56:29.521 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.11 2024-04-25T16:56:29.806 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 systemd[1]: Stopping Ceph osd.11 for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.809+0000 7fe1be4b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.809+0000 7fe1be4b0700 -1 osd.11 112 *** Got signal Terminated *** 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:29 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:29.809+0000 7fe1be4b0700 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:30.199 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:30.200 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:30.200 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:30.200 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:30 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:30.029+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:31.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:31 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:31.020+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:32.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:32.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:32.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:32.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:32.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:32.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:32.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.003+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:33.448 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:33.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:32 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:32.992+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6820 osd.8 since back 2024-04-25T16:56:09.338075+0000 front 2024-04-25T16:56:09.338107+0000 (oldest deadline 2024-04-25T16:56:32.838094+0000) 2024-04-25T16:56:34.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6806 osd.0 since back 2024-04-25T16:55:01.325582+0000 front 2024-04-25T16:55:01.325529+0000 (oldest deadline 2024-04-25T16:55:27.225535+0000) 2024-04-25T16:56:34.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6804 osd.1 since back 2024-04-25T16:55:11.227871+0000 front 2024-04-25T16:55:11.227583+0000 (oldest deadline 2024-04-25T16:55:35.927810+0000) 2024-04-25T16:56:34.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6804 osd.2 since back 2024-04-25T16:55:22.230028+0000 front 2024-04-25T16:55:22.230084+0000 (oldest deadline 2024-04-25T16:55:46.930197+0000) 2024-04-25T16:56:34.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6812 osd.4 since back 2024-04-25T16:55:37.134013+0000 front 2024-04-25T16:55:37.134025+0000 (oldest deadline 2024-04-25T16:56:03.034056+0000) 2024-04-25T16:56:34.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6812 osd.5 since back 2024-04-25T16:55:44.135098+0000 front 2024-04-25T16:55:44.135121+0000 (oldest deadline 2024-04-25T16:56:08.235186+0000) 2024-04-25T16:56:34.449 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.112:6822 osd.6 since back 2024-04-25T16:55:49.936212+0000 front 2024-04-25T16:55:49.936228+0000 (oldest deadline 2024-04-25T16:56:15.836199+0000) 2024-04-25T16:56:34.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.116:6820 osd.7 since back 2024-04-25T16:56:01.137470+0000 front 2024-04-25T16:56:01.137443+0000 (oldest deadline 2024-04-25T16:56:25.837321+0000) 2024-04-25T16:56:34.450 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:34 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11[58446]: 2024-04-25T16:56:34.000+0000 7fe1baa66700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.133:6820 osd.8 since back 2024-04-25T16:56:09.338075+0000 front 2024-04-25T16:56:09.338107+0000 (oldest deadline 2024-04-25T16:56:32.838094+0000) 2024-04-25T16:56:35.531 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:35 smithi133 podman[71221]: 2024-04-25 16:56:35.272416594 +0000 UTC m=+5.557307836 container died 75544e02ef48844e9c79063a900e51162946589e58d87276b6d0149b40de3457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:56:35.823 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:35 smithi133 podman[71221]: 2024-04-25 16:56:35.617272059 +0000 UTC m=+5.902163302 container cleanup 75544e02ef48844e9c79063a900e51162946589e58d87276b6d0149b40de3457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T16:56:35.823 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:35 smithi133 bash[71221]: ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11 2024-04-25T16:56:36.198 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:35 smithi133 podman[71238]: 2024-04-25 16:56:35.826780972 +0000 UTC m=+0.550391364 container remove 75544e02ef48844e9c79063a900e51162946589e58d87276b6d0149b40de3457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T16:56:36.543 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:36 smithi133 podman[71328]: 2024-04-25 16:56:36.138751004 +0000 UTC m=+0.017728647 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:56:36.544 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:36 smithi133 podman[71328]: 2024-04-25 16:56:36.252907476 +0000 UTC m=+0.131885145 container create 143b3eed072ec03811a0df5bcb163515235f47b20ab6155e260617c0cfe48306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:56:36.544 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:36 smithi133 podman[71328]: 2024-04-25 16:56:36.471576072 +0000 UTC m=+0.350553696 container init 143b3eed072ec03811a0df5bcb163515235f47b20ab6155e260617c0cfe48306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T16:56:36.544 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:36 smithi133 podman[71328]: 2024-04-25 16:56:36.47458731 +0000 UTC m=+0.353564935 container start 143b3eed072ec03811a0df5bcb163515235f47b20ab6155e260617c0cfe48306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T16:56:36.544 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:36 smithi133 podman[71328]: 2024-04-25 16:56:36.546482289 +0000 UTC m=+0.425459917 container attach 143b3eed072ec03811a0df5bcb163515235f47b20ab6155e260617c0cfe48306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T16:56:36.948 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:36 smithi133 podman[71328]: 2024-04-25 16:56:36.758971725 +0000 UTC m=+0.637949352 container died 143b3eed072ec03811a0df5bcb163515235f47b20ab6155e260617c0cfe48306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T16:56:37.301 INFO:journalctl@ceph.osd.11.smithi133.stdout:Apr 25 16:56:37 smithi133 podman[71354]: 2024-04-25 16:56:37.283045078 +0000 UTC m=+0.519283661 container remove 143b3eed072ec03811a0df5bcb163515235f47b20ab6155e260617c0cfe48306 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T16:56:37.306 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T16:56:37.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:37.342 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T16:56:37.342 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T16:56:37.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.a 2024-04-25T16:56:37.639 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:56:37 smithi112 systemd[1]: Stopping Ceph mds.a for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:38.094 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:56:37 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a[71875]: 2024-04-25T16:56:37.641+0000 7f38b27bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:38.094 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:56:37 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a[71875]: 2024-04-25T16:56:37.641+0000 7f38b27bc700 -1 mds.a *** got signal Terminated *** 2024-04-25T16:56:41.756 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:56:41 smithi112 podman[93056]: 2024-04-25 16:56:41.420985436 +0000 UTC m=+3.887774465 container died 8f1f0b22835a46ff01a867c006d1a08c79fb8e7e719b30397af986f2837fee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True) 2024-04-25T16:56:42.039 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:56:41 smithi112 podman[93056]: 2024-04-25 16:56:41.833458274 +0000 UTC m=+4.300247292 container cleanup 8f1f0b22835a46ff01a867c006d1a08c79fb8e7e719b30397af986f2837fee01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T16:56:42.039 INFO:journalctl@ceph.mds.a.smithi112.stdout:Apr 25 16:56:41 smithi112 bash[93056]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-a 2024-04-25T16:56:42.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T16:56:42.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:42.198 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T16:56:42.198 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T16:56:42.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.d 2024-04-25T16:56:42.344 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:42 smithi112 systemd[1]: Stopping Ceph mds.d for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:42.844 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d[70915]: 2024-04-25T16:56:42.520+0000 7ff94fe91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:42.844 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:42 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d[70915]: 2024-04-25T16:56:42.520+0000 7ff94fe91700 -1 mds.d *** got signal Terminated *** 2024-04-25T16:56:45.844 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:45 smithi112 podman[93178]: 2024-04-25 16:56:45.5398725 +0000 UTC m=+3.122926664 container died b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-25T16:56:46.318 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:45 smithi112 podman[93178]: 2024-04-25 16:56:45.970584259 +0000 UTC m=+3.553638409 container cleanup b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T16:56:46.318 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:45 smithi112 bash[93178]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d 2024-04-25T16:56:46.319 INFO:journalctl@ceph.mds.d.smithi112.stdout:Apr 25 16:56:46 smithi112 podman[93191]: 2024-04-25 16:56:46.129893243 +0000 UTC m=+0.584191838 container remove b9ba950d77be6bc95b7b585a84b7d86574345627e294bc1ae7854a47c29b88a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T16:56:46.337 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T16:56:46.370 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:46.370 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T16:56:46.371 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T16:56:46.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.g 2024-04-25T16:56:46.594 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:56:46 smithi112 systemd[1]: Stopping Ceph mds.g for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:47.094 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:56:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g[71519]: 2024-04-25T16:56:46.698+0000 7f2806acc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:47.094 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:56:46 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g[71519]: 2024-04-25T16:56:46.698+0000 7f2806acc700 -1 mds.g *** got signal Terminated *** 2024-04-25T16:56:50.844 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:56:50 smithi112 podman[93299]: 2024-04-25 16:56:50.582877416 +0000 UTC m=+3.988586807 container died 0361e0ebd79fe031babb65cf26cf1704084ff362b51a65c7124914c8b75e58fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T16:56:51.213 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:56:50 smithi112 podman[93299]: 2024-04-25 16:56:50.949244032 +0000 UTC m=+4.354953476 container cleanup 0361e0ebd79fe031babb65cf26cf1704084ff362b51a65c7124914c8b75e58fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T16:56:51.214 INFO:journalctl@ceph.mds.g.smithi112.stdout:Apr 25 16:56:50 smithi112 bash[93299]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-g 2024-04-25T16:56:51.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T16:56:51.402 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:51.403 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T16:56:51.403 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T16:56:51.403 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.j 2024-04-25T16:56:51.844 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:51 smithi112 systemd[1]: Stopping Ceph mds.j for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:51.844 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j[70569]: 2024-04-25T16:56:51.702+0000 7f13360b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:51.845 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:51 smithi112 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j[70569]: 2024-04-25T16:56:51.702+0000 7f13360b7700 -1 mds.j *** got signal Terminated *** 2024-04-25T16:56:55.066 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:54 smithi112 podman[93421]: 2024-04-25 16:56:54.777818859 +0000 UTC m=+3.161042346 container died c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T16:56:55.344 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:55 smithi112 podman[93421]: 2024-04-25 16:56:55.069470338 +0000 UTC m=+3.452693833 container cleanup c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:56:55.344 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:55 smithi112 bash[93421]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j 2024-04-25T16:56:55.344 INFO:journalctl@ceph.mds.j.smithi112.stdout:Apr 25 16:56:55 smithi112 podman[93435]: 2024-04-25 16:56:55.279114445 +0000 UTC m=+0.500956475 container remove c264df212309b86f609b86509d34545020c6a8b18cd4145581ef73535d010861 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T16:56:55.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T16:56:55.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:55.401 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T16:56:55.401 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T16:56:55.401 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.b 2024-04-25T16:56:55.706 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:56:55 smithi116 systemd[1]: Stopping Ceph mds.b for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:55.970 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:56:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: 2024-04-25T16:56:55.708+0000 7f45e7325700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:55.970 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:56:55 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b[63383]: 2024-04-25T16:56:55.708+0000 7f45e7325700 -1 mds.b *** got signal Terminated *** 2024-04-25T16:56:57.230 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:56:56 smithi116 podman[71646]: 2024-04-25 16:56:56.930568453 +0000 UTC m=+1.336861367 container died 755e27b715b3682a4a508b8e4ee17d93f3ef39cb6144182f75f55ca8a67a706e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T16:56:57.507 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:56:57 smithi116 podman[71646]: 2024-04-25 16:56:57.316824335 +0000 UTC m=+1.723117256 container cleanup 755e27b715b3682a4a508b8e4ee17d93f3ef39cb6144182f75f55ca8a67a706e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T16:56:57.507 INFO:journalctl@ceph.mds.b.smithi116.stdout:Apr 25 16:56:57 smithi116 bash[71646]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-b 2024-04-25T16:56:57.739 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T16:56:57.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:56:57.774 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T16:56:57.774 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T16:56:57.774 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.e 2024-04-25T16:56:58.095 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:56:57 smithi116 systemd[1]: Stopping Ceph mds.e for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:56:58.507 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:56:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: 2024-04-25T16:56:58.097+0000 7f6e29807700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:56:58.507 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:56:58 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e[62758]: 2024-04-25T16:56:58.097+0000 7f6e29807700 -1 mds.e *** got signal Terminated *** 2024-04-25T16:56:59.712 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:56:59 smithi116 podman[71779]: 2024-04-25 16:56:59.459607187 +0000 UTC m=+1.462455935 container died a56bba8a68dd3f6f886a37ec839c327810d4a79ea13c55c80b1088046dc8d2ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T16:57:00.007 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:56:59 smithi116 podman[71779]: 2024-04-25 16:56:59.805902338 +0000 UTC m=+1.808751130 container cleanup a56bba8a68dd3f6f886a37ec839c327810d4a79ea13c55c80b1088046dc8d2ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:57:00.007 INFO:journalctl@ceph.mds.e.smithi116.stdout:Apr 25 16:56:59 smithi116 bash[71779]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-e 2024-04-25T16:57:00.225 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T16:57:00.257 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:00.257 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T16:57:00.257 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T16:57:00.257 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.h 2024-04-25T16:57:00.581 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:57:00 smithi116 systemd[1]: Stopping Ceph mds.h for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:57:01.007 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:57:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h[63069]: 2024-04-25T16:57:00.584+0000 7efd16a9a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:57:01.007 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:57:00 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h[63069]: 2024-04-25T16:57:00.584+0000 7efd16a9a700 -1 mds.h *** got signal Terminated *** 2024-04-25T16:57:04.912 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:57:04 smithi116 podman[71900]: 2024-04-25 16:57:04.610760187 +0000 UTC m=+4.134961845 container died bd9c689a7e37b2e59adea4e63ec3f3e46492455d21a6d40cf4dcb750c96e09f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:57:05.232 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:57:05 smithi116 podman[71900]: 2024-04-25 16:57:05.000890151 +0000 UTC m=+4.525091806 container cleanup bd9c689a7e37b2e59adea4e63ec3f3e46492455d21a6d40cf4dcb750c96e09f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T16:57:05.232 INFO:journalctl@ceph.mds.h.smithi116.stdout:Apr 25 16:57:05 smithi116 bash[71900]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-h 2024-04-25T16:57:05.361 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T16:57:05.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:05.392 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T16:57:05.393 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T16:57:05.393 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.k 2024-04-25T16:57:05.507 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:57:05 smithi116 systemd[1]: Stopping Ceph mds.k for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:57:06.007 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:57:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: 2024-04-25T16:57:05.696+0000 7f2a4ab3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:57:06.007 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:57:05 smithi116 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k[62437]: 2024-04-25T16:57:05.696+0000 7f2a4ab3e700 -1 mds.k *** got signal Terminated *** 2024-04-25T16:57:07.560 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:57:07 smithi116 podman[72022]: 2024-04-25 16:57:07.294809078 +0000 UTC m=+1.686228752 container died 627db4aa9bea5cf11ee89a487f2357e2b6b18d25d19c3082981c4ea620be34c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T16:57:07.935 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:57:07 smithi116 podman[72022]: 2024-04-25 16:57:07.671285629 +0000 UTC m=+2.062705314 container cleanup 627db4aa9bea5cf11ee89a487f2357e2b6b18d25d19c3082981c4ea620be34c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5) 2024-04-25T16:57:07.936 INFO:journalctl@ceph.mds.k.smithi116.stdout:Apr 25 16:57:07 smithi116 bash[72022]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-k 2024-04-25T16:57:08.093 DEBUG:teuthology.orchestra.run.smithi116:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T16:57:08.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:08.123 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T16:57:08.123 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T16:57:08.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.c 2024-04-25T16:57:08.421 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:57:08 smithi133 systemd[1]: Stopping Ceph mds.c for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:57:08.678 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:57:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c[63451]: 2024-04-25T16:57:08.423+0000 7fba71f99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:57:08.679 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:57:08 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c[63451]: 2024-04-25T16:57:08.423+0000 7fba71f99700 -1 mds.c *** got signal Terminated *** 2024-04-25T16:57:12.696 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:57:12 smithi133 podman[71446]: 2024-04-25 16:57:12.440498678 +0000 UTC m=+4.119398794 container died c5cc1be113a3f2a855ff95999ba2a59b729bccbf5580ef5a392d4ba9f80a9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:57:12.948 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:57:12 smithi133 podman[71446]: 2024-04-25 16:57:12.77323323 +0000 UTC m=+4.452133347 container cleanup c5cc1be113a3f2a855ff95999ba2a59b729bccbf5580ef5a392d4ba9f80a9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T16:57:12.948 INFO:journalctl@ceph.mds.c.smithi133.stdout:Apr 25 16:57:12 smithi133 bash[71446]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-c 2024-04-25T16:57:13.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T16:57:13.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:13.235 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T16:57:13.235 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T16:57:13.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.f 2024-04-25T16:57:13.549 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:57:13 smithi133 systemd[1]: Stopping Ceph mds.f for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:57:13.948 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:57:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f[63140]: 2024-04-25T16:57:13.551+0000 7fd91d151700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:57:13.949 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:57:13 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f[63140]: 2024-04-25T16:57:13.551+0000 7fd91d151700 -1 mds.f *** got signal Terminated *** 2024-04-25T16:57:17.198 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:57:16 smithi133 podman[71581]: 2024-04-25 16:57:16.925292329 +0000 UTC m=+3.470809961 container died b1e19a565213f4983fd75466ffc85ef414d91190612d10b938a48f45570905a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:57:17.541 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:57:17 smithi133 podman[71581]: 2024-04-25 16:57:17.301757181 +0000 UTC m=+3.847274813 container cleanup b1e19a565213f4983fd75466ffc85ef414d91190612d10b938a48f45570905a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:57:17.541 INFO:journalctl@ceph.mds.f.smithi133.stdout:Apr 25 16:57:17 smithi133 bash[71581]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-f 2024-04-25T16:57:17.671 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T16:57:17.704 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:17.704 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T16:57:17.705 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T16:57:17.705 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.i 2024-04-25T16:57:17.948 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:17 smithi133 systemd[1]: Stopping Ceph mds.i for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:57:18.448 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i[63759]: 2024-04-25T16:57:18.046+0000 7fe6bf82c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:57:18.448 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:18 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i[63759]: 2024-04-25T16:57:18.046+0000 7fe6bf82c700 -1 mds.i *** got signal Terminated *** 2024-04-25T16:57:21.198 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:20 smithi133 podman[71703]: 2024-04-25 16:57:20.913413802 +0000 UTC m=+2.991480261 container died a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:57:21.544 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:21 smithi133 podman[71703]: 2024-04-25 16:57:21.363755609 +0000 UTC m=+3.441822059 container cleanup a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:57:21.545 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:21 smithi133 bash[71703]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i 2024-04-25T16:57:21.786 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T16:57:21.807 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:21 smithi133 podman[71717]: 2024-04-25 16:57:21.548040981 +0000 UTC m=+0.628049644 container remove a2a3658ee3f0718635d8b8dea1c89cac64888a1f5e8b27b965750b6d725e5a33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T16:57:21.807 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:21 smithi133 systemd[1]: ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.i.service: Deactivated successfully. 2024-04-25T16:57:21.807 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:21 smithi133 systemd[1]: Stopped Ceph mds.i for a7905982-0320-11ef-bc93-c7b262605968. 2024-04-25T16:57:21.807 INFO:journalctl@ceph.mds.i.smithi133.stdout:Apr 25 16:57:21 smithi133 systemd[1]: ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.i.service: Consumed 1min 41.131s CPU time. 2024-04-25T16:57:21.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:21.817 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T16:57:21.818 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T16:57:21.818 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.l 2024-04-25T16:57:22.122 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:57:21 smithi133 systemd[1]: Stopping Ceph mds.l for a7905982-0320-11ef-bc93-c7b262605968... 2024-04-25T16:57:22.448 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:57:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l[62835]: 2024-04-25T16:57:22.125+0000 7f3950976700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:57:22.448 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:57:22 smithi133 ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l[62835]: 2024-04-25T16:57:22.125+0000 7f3950976700 -1 mds.l *** got signal Terminated *** 2024-04-25T16:57:25.199 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:57:24 smithi133 podman[71824]: 2024-04-25 16:57:24.918422611 +0000 UTC m=+2.881993870 container died 13692d82e059f43c61cc07f71241e1a2a462952e905833ca710c57f5656db700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T16:57:25.530 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:57:25 smithi133 podman[71824]: 2024-04-25 16:57:25.316788302 +0000 UTC m=+3.280359548 container cleanup 13692d82e059f43c61cc07f71241e1a2a462952e905833ca710c57f5656db700 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T16:57:25.531 INFO:journalctl@ceph.mds.l.smithi133.stdout:Apr 25 16:57:25 smithi133 bash[71824]: ceph-a7905982-0320-11ef-bc93-c7b262605968-mds-l 2024-04-25T16:57:25.659 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a7905982-0320-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T16:57:25.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:57:25.690 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T16:57:25.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7905982-0320-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T16:57:25.880 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:57:28.261 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7905982-0320-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T16:57:28.447 INFO:teuthology.orchestra.run.smithi116.stdout:Deleting cluster with fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:57:30.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7905982-0320-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T16:57:31.116 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T16:57:33.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:57:33.478 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:57:33.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:57:33.535 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T16:57:33.537 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi112/crash 2024-04-25T16:57:33.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash -- . 2024-04-25T16:57:33.574 INFO:teuthology.orchestra.run.smithi112.stderr:tar: /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T16:57:33.574 INFO:teuthology.orchestra.run.smithi112.stderr:tar: Error is not recoverable: exiting now 2024-04-25T16:57:33.576 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi116/crash 2024-04-25T16:57:33.577 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash -- . 2024-04-25T16:57:33.619 INFO:teuthology.orchestra.run.smithi116.stderr:tar: /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T16:57:33.620 INFO:teuthology.orchestra.run.smithi116.stderr:tar: Error is not recoverable: exiting now 2024-04-25T16:57:33.622 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi133/crash 2024-04-25T16:57:33.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash -- . 2024-04-25T16:57:33.666 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/a7905982-0320-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T16:57:33.666 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2024-04-25T16:57:33.667 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T16:57:33.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T16:57:33.701 INFO:tasks.cephadm:Compressing logs... 2024-04-25T16:57:33.701 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:57:33.744 DEBUG:teuthology.orchestra.run.smithi116:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:57:33.746 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:57:33.766 INFO:teuthology.orchestra.run.smithi112.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-04-25T16:57:33.767 INFO:teuthology.orchestra.run.smithi112.stderr: -- /var/log/ceph/cephadm.log 2024-04-25T16:57:33.767 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log 2024-04-25T16:57:33.767 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T16:57:33.767 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T16:57:33.769 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T16:57:33.769 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T16:57:33.769 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T16:57:33.770 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T16:57:33.770 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:: No such file or directory 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5/var/log/ceph/cephadm.log: --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T16:57:33.780 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.h.log: --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T16:57:33.781 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T16:57:33.784 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.x.log: 87.1% 90.4% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T16:57:33.784 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T16:57:33.784 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.6.log: 94.6% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr: 94.7% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.4% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T16:57:33.785 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T16:57:33.786 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T16:57:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T16:57:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:: No such file or directory 2024-04-25T16:57:33.787 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T16:57:33.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.2% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr: 90.5% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.i.log: 92.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr: 84.1% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T16:57:33.789 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T16:57:33.790 INFO:teuthology.orchestra.run.smithi112.stderr: 91.1% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T16:57:33.808 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T16:57:33.835 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T16:57:33.881 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.b.log: /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.k.log: 91.0% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T16:57:33.948 INFO:teuthology.orchestra.run.smithi116.stderr: 92.2% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T16:57:33.983 INFO:teuthology.orchestra.run.smithi133.stderr: 94.5% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T16:57:34.027 INFO:teuthology.orchestra.run.smithi133.stderr: 94.5% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T16:57:34.204 INFO:teuthology.orchestra.run.smithi112.stderr: 90.0% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T16:57:34.237 INFO:teuthology.orchestra.run.smithi116.stderr: 84.2% 92.1% 87.1% -- replaced with /var/log/ceph/cephadm.log.gz -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.cephadm.log.gz -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T16:57:34.237 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-04-25T16:57:34.237 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-04-25T16:57:34.287 INFO:teuthology.orchestra.run.smithi116.stderr: 90.5% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T16:57:34.875 INFO:teuthology.orchestra.run.smithi112.stderr: 90.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T16:57:35.208 INFO:teuthology.orchestra.run.smithi133.stderr: 93.2% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T16:57:37.946 INFO:teuthology.orchestra.run.smithi133.stderr: 94.2% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T16:57:41.430 INFO:teuthology.orchestra.run.smithi116.stderr: 91.9% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T16:57:44.824 INFO:teuthology.orchestra.run.smithi133.stderr: 96.0% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T16:57:51.815 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T16:57:56.867 INFO:teuthology.orchestra.run.smithi112.stderr: 91.6% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T16:57:57.510 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T16:57:58.514 INFO:teuthology.orchestra.run.smithi112.stderr: 91.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T16:57:59.757 INFO:teuthology.orchestra.run.smithi112.stderr: 91.5% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T16:57:59.759 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T16:57:59.759 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m26.003s 2024-04-25T16:57:59.760 INFO:teuthology.orchestra.run.smithi112.stderr:user 1m30.542s 2024-04-25T16:57:59.760 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m2.770s 2024-04-25T16:57:59.844 INFO:teuthology.orchestra.run.smithi133.stderr: 91.4% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T16:58:01.093 INFO:teuthology.orchestra.run.smithi133.stderr: 91.2% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T16:58:06.806 INFO:teuthology.orchestra.run.smithi133.stderr: 91.2% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T16:58:06.808 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-04-25T16:58:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m33.048s 2024-04-25T16:58:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:user 2m3.043s 2024-04-25T16:58:06.809 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m3.684s 2024-04-25T16:59:28.075 INFO:teuthology.orchestra.run.smithi116.stderr: 95.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T17:00:21.681 INFO:teuthology.orchestra.run.smithi116.stderr: 95.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T17:01:21.716 INFO:teuthology.orchestra.run.smithi116.stderr: 91.4% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T17:01:34.677 INFO:teuthology.orchestra.run.smithi116.stderr: 96.0% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T17:03:04.368 INFO:teuthology.orchestra.run.smithi116.stderr: 91.6% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T17:03:50.692 INFO:teuthology.orchestra.run.smithi116.stderr: 91.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T17:04:36.834 INFO:teuthology.orchestra.run.smithi116.stderr: 91.3% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T17:10:33.981 INFO:teuthology.orchestra.run.smithi116.stderr: 93.5% -- replaced with /var/log/ceph/a7905982-0320-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T17:10:34.079 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-04-25T17:10:34.079 INFO:teuthology.orchestra.run.smithi116.stderr:real 13m0.315s 2024-04-25T17:10:34.079 INFO:teuthology.orchestra.run.smithi116.stderr:user 7m40.537s 2024-04-25T17:10:34.079 INFO:teuthology.orchestra.run.smithi116.stderr:sys 0m20.092s 2024-04-25T17:10:34.080 INFO:tasks.cephadm:Archiving logs... 2024-04-25T17:10:34.080 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi112/log 2024-04-25T17:10:34.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T17:10:50.062 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi116/log 2024-04-25T17:10:50.063 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T17:12:04.066 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi133/log 2024-04-25T17:12:04.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T17:12:23.224 INFO:tasks.cephadm:Removing cluster... 2024-04-25T17:12:23.224 DEBUG:teuthology.orchestra.run.smithi112:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7905982-0320-11ef-bc93-c7b262605968 --force 2024-04-25T17:12:23.415 INFO:teuthology.orchestra.run.smithi112.stdout:Deleting cluster with fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T17:12:23.808 DEBUG:teuthology.orchestra.run.smithi116:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7905982-0320-11ef-bc93-c7b262605968 --force 2024-04-25T17:12:24.717 INFO:teuthology.orchestra.run.smithi116.stdout:Deleting cluster with fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T17:12:25.374 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a7905982-0320-11ef-bc93-c7b262605968 --force 2024-04-25T17:12:25.567 INFO:teuthology.orchestra.run.smithi133.stdout:Deleting cluster with fsid: a7905982-0320-11ef-bc93-c7b262605968 2024-04-25T17:12:25.990 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T17:12:25.990 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T17:12:26.009 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T17:12:26.038 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T17:12:26.053 INFO:tasks.cephadm:Teardown complete 2024-04-25T17:12:26.054 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T17:12:26.065 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T17:12:26.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T17:12:26.067 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T17:12:26.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T17:12:26.209 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T17:12:26.220 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T17:12:26.220 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T17:12:26.223 DEBUG:teuthology.orchestra.run.smithi116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T17:12:26.225 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T17:12:26.237 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2024-04-25T17:12:26.239 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 1: ntpq: command not found 2024-04-25T17:12:26.239 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-04-25T17:12:26.241 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T17:12:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2024-04-25T17:12:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:^* hv01.front.sepia.ceph.com 3 6 377 42 +755us[ +757us] +/- 55ms 2024-04-25T17:12:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -1359us[-1357us] +/- 61ms 2024-04-25T17:12:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +631us[ +633us] +/- 78ms 2024-04-25T17:12:26.242 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T17:12:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T17:12:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-04-25T17:12:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 +766us[ +747us] +/- 55ms 2024-04-25T17:12:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -1356us[-1376us] +/- 61ms 2024-04-25T17:12:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +652us[ +632us] +/- 78ms 2024-04-25T17:12:26.244 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T17:12:26.321 INFO:teuthology.orchestra.run.smithi116.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T17:12:26.322 INFO:teuthology.orchestra.run.smithi116.stdout:=============================================================================== 2024-04-25T17:12:26.322 INFO:teuthology.orchestra.run.smithi116.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +755us[ +748us] +/- 55ms 2024-04-25T17:12:26.322 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -1363us[-1363us] +/- 61ms 2024-04-25T17:12:26.322 INFO:teuthology.orchestra.run.smithi116.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +632us[ +632us] +/- 78ms 2024-04-25T17:12:26.322 INFO:teuthology.orchestra.run.smithi116.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T17:12:26.339 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T17:12:26.351 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T17:12:26.351 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T17:12:26.403 DEBUG:teuthology.orchestra.run.smithi112:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T17:12:26.481 DEBUG:teuthology.orchestra.run.smithi116:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T17:12:26.580 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T17:12:26.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T17:12:26.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:12:26.674 DEBUG:teuthology.orchestra.run.smithi116:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T17:12:26.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:12:26.703 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T17:12:26.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:12:26.729 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T17:12:26.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T17:12:26.774 INFO:teuthology.task.internal:Duration was 3058.026234 seconds 2024-04-25T17:12:26.774 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T17:12:26.797 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T17:12:26.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T17:12:26.801 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T17:12:26.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T17:12:26.839 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T17:12:26.840 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T17:12:26.852 INFO:teuthology.orchestra.run.smithi116.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T17:12:27.343 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T17:12:27.343 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T17:12:27.344 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T17:12:27.367 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi116.front.sepia.ceph.com 2024-04-25T17:12:27.367 DEBUG:teuthology.orchestra.run.smithi116:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T17:12:27.390 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-04-25T17:12:27.391 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T17:12:27.422 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T17:12:27.422 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T17:12:27.425 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T17:12:27.433 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T17:12:27.487 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T17:12:27.487 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T17:12:27.495 DEBUG:teuthology.orchestra.run.smithi116:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T17:12:27.511 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T17:12:27.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T17:12:27.879 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T17:12:27.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T17:12:27.903 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T17:12:27.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T17:12:27.952 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T17:12:27.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:12:27.967 DEBUG:teuthology.orchestra.run.smithi116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:12:27.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:12:27.986 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T17:12:28.001 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = core 2024-04-25T17:12:28.013 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-04-25T17:12:28.026 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:12:28.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:12:28.053 DEBUG:teuthology.orchestra.run.smithi116:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:12:28.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:12:28.076 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T17:12:28.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T17:12:28.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T17:12:28.102 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T17:12:28.103 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi112 2024-04-25T17:12:28.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T17:12:28.146 DEBUG:teuthology.misc:Transferring archived files from smithi116:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi116 2024-04-25T17:12:28.146 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T17:12:28.181 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263/remote/smithi133 2024-04-25T17:12:28.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T17:12:28.216 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T17:12:28.217 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T17:12:28.222 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T17:12:28.224 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T17:12:28.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T17:12:28.278 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T17:12:28.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T17:12:28.301 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T17:12:28.301 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T17:12:28.304 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T17:12:28.306 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T17:12:28.316 INFO:teuthology.orchestra.run.smithi112.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 17:12 /home/ubuntu/cephtest 2024-04-25T17:12:28.318 INFO:teuthology.orchestra.run.smithi116.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 17:12 /home/ubuntu/cephtest 2024-04-25T17:12:28.320 INFO:teuthology.orchestra.run.smithi133.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 17:12 /home/ubuntu/cephtest 2024-04-25T17:12:28.321 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T17:12:28.414 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T17:12:28.519 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T17:12:28.546 DEBUG:teuthology.nuke:shortname: smithi112 2024-04-25T17:12:28.546 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T17:12:28.572 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:15:57.649947', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHURfvh9sUpbhyGvSS0JiGGR/IYYvEWYs4eJ/el3XPz2AokYBYTycIZUenLNyCXK7GfuvBonxia57zMHPEBOR+8='} 2024-04-25T17:12:28.576 DEBUG:teuthology.nuke:shortname: smithi116 2024-04-25T17:12:28.576 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T17:12:28.598 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:15:57.647323', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDBtNxWV7Iqte/HfqVxNhwBQ+C7WeEL3bN2y4Q6AVWwnLT2puphE0CJQUot2PZRoZw8RBfm5nJVxSSDr9FL10wk='} 2024-04-25T17:12:28.602 DEBUG:teuthology.nuke:shortname: smithi133 2024-04-25T17:12:28.602 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T17:12:28.626 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673263', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 16:15:57.648732', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGXKQDrVO9Nh0jJ0v/bl77wWY9eodLl4k0MyzB4T9cMkGGP/YWoKRo6TTnySXkQv4FvTWd2me7hjQZ30/RgxCzA='} 2024-04-25T17:12:28.635 INFO:teuthology.orchestra.console.smithi112:Power off 2024-04-25T17:12:28.635 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T17:12:28.651 INFO:teuthology.orchestra.console.smithi116:Power off 2024-04-25T17:12:28.652 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T17:12:28.662 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-04-25T17:12:28.662 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:28.677 DEBUG:teuthology.orchestra.console.smithi116:power off output: Chassis Power Control: Down/Off 2024-04-25T17:12:28.677 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:28.688 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T17:12:28.704 INFO:teuthology.orchestra.console.smithi133:Power off 2024-04-25T17:12:28.704 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T17:12:28.713 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-04-25T17:12:28.726 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2024-04-25T17:12:28.727 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:28.749 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-25T17:12:32.689 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:32.715 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:32.750 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:32.800 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T17:12:32.827 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is on 2024-04-25T17:12:32.860 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-04-25T17:12:36.805 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:36.828 DEBUG:teuthology.orchestra.console.smithi116:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:36.861 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T17:12:36.916 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-04-25T17:12:36.940 DEBUG:teuthology.orchestra.console.smithi116:check power output: Chassis Power is off 2024-04-25T17:12:36.971 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2024-04-25T17:12:37.018 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-04-25T17:12:37.040 INFO:teuthology.orchestra.console.smithi116:Power off completed 2024-04-25T17:12:37.072 INFO:teuthology.orchestra.console.smithi133:Power off completed 2024-04-25T17:12:37.179 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 3058.0262343883514 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi112 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=551663228469486bbbc00d4e86b7a1a9 status: fail success: false 2024-04-25T17:12:37.180 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T17:12:37.263 INFO:teuthology.run:FAIL